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Abstract 

Health  monitoring  systems  have  demonstrated  the  ability  to  detect  potential 
failures  in  components  and  predict  how  long  until  a  critical  failure  is  likely  to  occur.  The 
decision  to  implement  these  systems  on  fielded  structures,  aircraft,  or  other  vehicles  is 
often  a  struggle  due  to  the  difficulty  of  demonstrating  to  prove  cost  savings  or  operational 
improvements  beyond  improved  safety.  A  system  architecture  to  identity  how  the  health 
monitoring  systems  are  integrated  into  fielded  aircraft  is  developed  to  assess  cost, 
operations,  maintenance,  and  logistics  trade-spaces.  The  efficiency  of  a  health  monitoring 
system  is  examined  for  impacts  to  the  operation  of  a  squadron  of  cargo  aircraft  revealing 
sensitivity  to,  and  tolerance  for,  false  alarms  as  a  key  factor  in  total  system  performance. 
The  research  focuses  on  the  impacts  of  system-wide  changes  to  several  key  metrics: 
materiel  availability,  materiel  reliability,  ownership  cost,  and  mean  downtime.  Changes 
to  these  system-wide  variables  include:  diagnostic  and  prognostic  error,  false  alarm 
sensitivity,  supply  methods  and  timing,  maintenance  manning,  and  maintenance  repair 
window.  Potential  cost  savings  in  maintenance  and  logistics  processes  are  identified  as 
well  as  increases  in  operational  availability.  The  result  of  this  research  is  the  development 
of  a  tool  to  conduct  trade-space  analyses  regarding  the  effects  of  health  monitoring 
techniques  on  system  performance  and  operations  and  maintenance  costs. 
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INTEGRATED  SYSTEMS  HEALTH  MANAGEMENT  AS  AN  ENABLER  FOR 


CONDITION  BASED  MAINTENANCE  AND  AUTONOMIC  LOGISTICS 

I.  Introduction 

The  state  of  health  for  major  infrastructure  and  transportation  systems  nationally 
and  globally  is  deteriorating.  Aircraft,  in  particular,  are  an  example  where  lengthening 
service  lives  and  budget  constraints  can  affect  the  safety  of  the  vehicle  and  the  occupants. 
Currently,  more  frequent  inspections  are  required  as  service  life  increases  to  ensure  safety 
of  the  users  and  the  environment.  The  I-35W  Mississippi  River  bridge  collapse  in 
Minneapolis  in  2007,  for  instance,  resulted  in  loss  of  life  and  drove  a  massive  inspection 
cycle  for  United  States  highway  infrastructure  (Modares  &  Waksmanski,  2012). 
Maintenance  strategies  must  change  to  meet  the  extended  in-service  requirements  and  the 
constraints  imposed  by  shrinking  government  and  industry  budgets. 

Across  many  industries,  systems  are  exceeding  their  intended  design  lives, 
whether  they  are  ships,  bridges  or  US  Air  Force  aircraft.  Structural  health  monitoring 
(SHM)  research  and  application  has  the  potential  to  lengthen  the  service  life  of  a  range  of 
systems  and,  in  some  cases,  predict  failure  modes  and  times.  As  such,  funding  and 
research  are  focused  on  researching  new  technologies  and  applications.  However,  the 
cost  of  large  scale  implementation  in  the  case  of  hundreds  of  aircraft  or  approximately 
70000  “structurally  deficient”  highway  bridges  is  a  significant  hurdle  to  overcome  in 
most  instances  (Shoup,  Donohue,  &  Lang,  2011).  The  impact  of  shrinking  budgets  can 
also  reduce  inspection  frequency  or  delay  needed  repairs  in  favor  of  only  performing 
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mission  critical  tasks  (Roach,  2009).  Condition  based  maintenance  (CBM)  is  an  evolving 
maintenance  concept  with  a  goal  of  reducing  maintenance  and  thus  life  cycle  costs  while 
increasing  operational  availability.  It  is  made  possible,  in  part,  by  leveraging  health 
monitoring  techniques.  Integrated  system  health  management  (ISHM)  incorporates  health 
monitoring  functions  across  a  platform  to  provide  system- wide  state  of  health  diagnostics 
and  prognostics.  The  impact  of  ISHM  and  CBM  on  perfonnance,  cost,  and  supply  chain 
as  well  as  traditional  maintenance  inspections  and  practices  is  the  focus  of  this  research. 

With  the  F-35  maintenance  and  logistics  alone  projected  to  cost  $1.1  trillion  over 
the  5  5 -year  life  span  amid  shrinking  defense  budgets,  the  need  to  reduce  the  life  cycle 
cost  (LCC)  of  military  aircraft  is  paramount  (Shalal-Esa,  2013).  Legacy  aircraft  may  not 
be  fitted  with  the  proper  sensors  to  fully  implement  health  assessment  leading  to  costly 
inspections,  in  both  time  and  maintenance  dollars.  The  inspections  result  in  reduced 
operational  availability  (A0)  and  budget  available  for  other  needs.  Figure  1  depicts  the 
operational  concept  of  a  condition  based  maintenance  system  enabled  through  integrated 
systems  health  management. 

One  method  for  improving  operational  availability  through  the  implementation  of 
integrated  health  monitoring  is  by  using  data  collected  to  forecast  and  group  maintenance 
tasks  to  reduce  total  downtime.  This  reduction  can  be  realized  through  the  elimination  of 
multiple  set-up,  tear-down,  and  reassembly  cycles  in  favor  of  a  single  cycle  with 
numerous  maintenance  actions  accomplished  during  that  downtime.  Further  efficiencies 
may  be  realized  through  the  scheduling  of  maintenance  personnel  based  on  knowing 
when  and  what  repairs  are  required  prior  to  beginning  any  maintenance  activity. 
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Figure  1.  ISFIM/CBM  Operational  Concept 


Mean  downtime  (MDT)  is  a  measure  of  the  efficiency  of  maintenance  and 
logistics  processes.  MDT  is  the  total  time  from  when  the  element  of  a  system  is  taken  out 
of  service  until  it  is  again  declared  fully  mission  capable,  as  opposed  to  mean  time  to 
repair  which  is  the  hands-on  maintenance  repair  portion.  The  DoD  predicts  that  by 
implementing  CBM  MDT  will  be  “significantly  reduced”  by  perfonning  demand  versus 
time-driven  maintenance  (Under  Secretary  of  Defense  (AT&L),  May  2008).  Research  by 
Derriso  (2013),  however,  shows  the  MDT  may  actually  increase  as  the  system  will  only 
be  taken  down  for  maintenance  rather  than  additional  time-based  inspections,  which  are 
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often  short  in  duration.  He  also  observes  that  total  downtime  is  reduced  for  the  ISHM 
versus  the  baseline  time -based  case. 

What  is  missing  from  the  mean  and  total  downtime  measures  is  the  value  of  those 
times.  Is  the  time  being  spent  troubleshooting  symptoms,  locating  a  fault,  or  waiting  on 
parts,  as  in  the  baseline  case?  If  the  time  is  spent  only  on  repair  of  the  system  because  the 
ISHM/CBM  system  predicted  and  isolated  the  failure  as  well  as  ordered  parts  to  meet  the 
demand,  that  time  can  be  thought  of  has  having  more  quality  behind  it.  This  downtime 
quality  reflects  added  benefit  provided  by  the  ISHM/CBM  system. 

Problem  Statement 

While  health  monitoring  techniques  continue  to  evolve,  the  capability  to  study 
their  cost  and  availability  impacts  across  flight,  maintenance,  and  logistics  realms 
remains  a  difficult  task. 


Research  Objective 

Even  with  a  policy  that  requires  its  implementation,  CBM  has  to  “ buy  its  way” 
into  the  program.  Service  leadership  and  the  program  and  support  managers 
want  to  do  the  right  thing  for  the  warfighter,  but  a  return  on  the  investment  must 
be  identified  and  justified.  In  the  long  run,  any  Service  effort  to  develop  and 
deploy  CBM  should  be  leveraged  by  other  platforms  and  programs  (Under 
Secretary  of  Defense  (AT&L),  May  2008,  p.  1-7). 

This  research  seeks  to  detennine  the  design  and  implementation  processes  to 
thoroughly  integrate  ISHM,  CBM  and  logistics  systems  to  define  operational  and  cost 
trade-spaces  for  multiple  systems  with  multiple  subcomponents  in  each  system. 
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These  examinations  require  a  comprehensive  model  that  explores  the 
interconnections  between  multiple  systems  each  having  numerous  components  that  can 
provide  greater  detail  on  the  impacts  of  changes  in  one  area  propagating  throughout  the 
remainder  of  the  model.  This  research  focuses  on  the  potential  benefits  to  Ao  and 
operations  and  maintenance  (O&M)  cost  through  the  use  of  ISHM  systems  to  enable 
CBM.  This  in-turn  provides  a  detailed  examination  of  trade-offs  available  to  the  user. 
Providing  a  means  to  explore  Ao  and  cost  impact  with  the  opportunity  to  improve  both 
and  reduce  manpower  requirements  is  of  great  interest  to  program  managers,  system 
operators,  and  financial  planners 


Investigative  Questions 

The  questions  posed  in  this  dissertation  include  the  following: 

1 .  What  are  the  key  cost  and  effectiveness  drivers  for  an  ISHM  enabled  CBM 
and  autonomic  logistics  process? 

2.  What  is  a  reasonable  and  appropriate  scope  for  model  development  to 
establish  performance  requirements  for  ISHM  sensors  and  prognostics, 
maintenance,  and  logistic  processes? 

3.  What  are  the  operations  and  maintenance  cost  impacts  of  an  ISHM  enabled 
CBM  system? 

4.  Is  mean  downtime  a  good  measure  of  system  perfonnance  for  ISHM/CBM 
systems? 

5.  Does  maintenance  grouping  based  on  prognostics  improve  operational 
availability,  total  downtime  and  cost? 


Methodology  Overview 

The  approach  taken  in  this  research  builds  upon  prior  work  on  health  monitoring, 
condition  based  maintenance  and  logistics.  An  Arena1*  model  is  developed  incorporating 
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systems  architecture  principles  to  analyze  cost  and  availability  impacts  of  proposed 
concepts  of  operation.  A  baseline  (current)  case  is  compared  to  the  same  aircraft  with  a 
health  monitoring  system  implemented.  Analysis  of  support  levels,  both  maintenance  and 
supply,  are  examined  in  conjunction  with  health  monitoring  techniques. 

The  first  step  in  creating  the  model  is  to  determine  both  the  baseline  and 
ISHM/CBM  designs,  which  are  found  in  system  architectures  presented  in  chapters  3  and 
4.  The  inputs  required  and  relevant  for  each  process  are  identified  from  this  architecture. 
Defining  the  reliability  and  maintainability  of  the  system  is  next  with  collection  and 
utilization  methods  discussed  later.  Next,  the  processes  are  modeled  and  output 
parameters  required  for  the  measures  of  effectiveness  and  measures  of  perfonnance  are 
evaluated.  The  model  captures  accumulated  effects  on  system  perfonnance  and  aircraft 
life.  These  outputs  are  then  analyzed  to  determine  the  impacts  of  ISHM/CBM  on  the 
system  as  a  whole. 

A  baseline  model,  reflecting  current  maintenance  and  supply  policies  and 
capabilities  is  the  first  step  for  validation  of  model  performance  and  to  provide  a  starting 
point  for  system  performance.  Building  upon  the  baseline  “as-is”  model,  the  inclusion  of 
health  monitoring  and  prognostics  augments  or  replaces  current  maintenance  inspection 
and  preventive  replacement  procedures.  Utilizing  the  prognostic  capabilities  of  the  health 
monitoring  system,  infonnation  about  impending  failures  and  system  requirements  is  fed 
into  the  supply  system  to  attempt  to  match  the  aircraft  need  with  logistics  capabilities. 
Based  upon  logistics,  maintenance,  and  operational  requirements,  a  time  frame  to 
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maintain  the  aircraft  is  projected  to  balance  these  requirements  with  the  goal  of  keeping 
operational  availability  high  and  cost  low. 

The  aircraft  under  study  in  this  research  is  the  C-17  Globemaster  III.  These 
aircraft  are  currently  fully  fielded  in  the  USAF  inventory  and  they  are  planned  to  be  in 
the  fleet  for  a  long  duration.  The  C-17  employs  CBM  techniques  to  an  extent,  with 
limited  built  in  test  capabilities  and  diagnostics  (Smith,  2003).  While  particular  aircraft 
subsystems  represented  in  the  model  are  simulated,  they  do  not  represent  actual  aircraft 
systems.  Therefore,  this  model  can  also  be  readily  applied  more  broadly  to  USAF  cargo 
aircraft  not  just  the  C-17.  Mission  profiles  and  preflight  activities  utilized  are 
representative  of  transport  aircraft  operations  and  yearly  flying  hours  are  comparable  to 
the  C-17.  This  model  will  process  a  squadron  of  12  aircraft  through  current  operational 
activities  and  compare  those  results  to  the  same  squadron  with  heath  monitoring, 
condition  based  maintenance  and  a  supply  system  interwoven  with  these  advanced 
techniques. 

Assumptions/Limitations 

Assumptions  in  this  research  are  done  in  an  effort  to  limit  the  scope  and 
complexity  of  model.  While  further  assumptions  are  contained  in  chapters  3-7,  those 
relevant  to  the  entire  work  are: 

•  A  squadron  of  12  aircraft  is  used  for  modeling  and  analysis; 

•  All  components  of  interest  have  monitoring  sensors; 

•  All  components  are  non-repairable,  that  is,  replaced  versus  repaired  when 
required; 
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•  Performance  impact  to  the  aircraft  from  weight  and  power  requirements  of  the 
ISHM  system  and  sensors  is  negligible; 

•  One  subsystem  comprised  of  20  unique  components  is  modeled; 

•  Inflation  is  not  considered; 

•  Military  deployments  are  ignored; 

•  Personnel  are  devoted  to  the  aircraft  in  the  model; 

•  Only  direct  maintenance  actions  required  for  the  components  under  study  are 
recorded. 

Limitations  of  this  work  lie  in  that  capturing  costs  savings  may  not  be  directly 
possible  for  military  systems  due  to  deployments,  allocation  of  personnel  across  multiple 
systems  and  the  mix  of  uniformed  and  contractor  personnel 


Implications 

This  research  model  has  the  potential  to  not  only  reduce  aircraft  O&M  costs,  but 
also  to  improve  availability  and  reduce  maintenance  manning  and/or  workload.  This 
work  allows  for  the  visualization  of  system  processes  and  the  impact  of  health  monitoring 
techniques  on  overall  performance.  Further,  the  methods  and  techniques  used  herein  are 
applicable  to  other  vehicle  types  and  systems. 

A  key  impact  of  this  work  is  the  ability  to  analyze  the  effects  of  new  health 
monitoring  techniques  by  inputting  them  into  the  model  and  studying  the  performance 
outcomes.  Alternatively,  if  a  desired  aircraft  or  logistics  performance  level  is  established, 
a  trade  study  can  be  performed  to  determine  health  monitoring  or  prognostics 
requirements  necessary  to  meet  those  demands.  In  general,  this  research  provides  for  the 
analysis  of  ISHM  enabled  CBM  and  supply  strategies  for  military  system  with  emphasis 
on  cost  and  availability  impacts. 
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This  work  demonstrates  that  it  is  possible,  based  on  a  limited  set  of  components, 
for  significant  savings  to  be  realized  in  the  maintenance  and  logistics  direct  costs  and 
through  a  reduction  in  required  personnel.  Additionally,  increased  availability  of  aircraft 
allows  for  fewer  aircraft  to  do  the  same  mission. 

Preview 

This  dissertation  is  organized  as  follows.  Chapter  2  provides  a  review  of  literature 
relevant  to  this  research  and  identifies  gaps  that  this  work  studies.  Chapter  3  establishes  a 
system  architecture  and  measures  of  effectiveness  for  model  evaluation.  Chapters  4 
through  6  are  drawn  from  manuscripts  published,  in  review,  or  to  be  submitted.  Chapter 
4,  titled  “Prognostic  Uncertainty,”  addresses  investigative  questions  #1  and  #2  identified 
above.  This  chapter  has  been  published  in  The  International  Journal  of  Prognostics  and 
Health  Management  with  the  article  titled  Impact  of  Prognostic  Uncertainty  in  System 
Health  Monitoring.  Chapter  5,  titled  “Component  Supply  Methods,”  explores  the  effect 
of  a  health  monitoring  system  on  supply  methodologies  and  addresses  questions  #1,  #2 
and  #3  identified  above.  This  chapter  is  drawn  from  a  manuscript  titled  Health 
Monitoring  Impact  on  Non-Reyairable  Component  Supply  Methods  that  will  be  submitted 
to  Decision  Sciences.  Chapter  6,  titled  “Maintenance  Manning  Processes,”  addresses 
research  questions  #1  through  #4.  The  text  is  from  a  manuscript  titled  An  Examination  of 
System  Health  Monitoring  Impact  on  Non-Repair  able  Component  Maintenance  Manning 
that  will  be  submitted  to  the  Journal  of  Quality  in  Maintenance  Engineering.  Chapter  7 
examines  investigative  questions  #4  and  #5  with  an  exploration  of  the  grouping  of 
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component  replacements.  Finally,  chapter  8  provides  overall  research  conclusions  and 
rec  ommendations . 
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II.  Literature  Review 


Chapter  Overview 

This  chapter  provides  background  information  on  key  concepts  and  techniques 
within  the  health  monitoring,  maintenance  and  supply  realms.  These  topics  are  addressed 
in  this  chapter  in  an  effort  to  identify  gaps  in  current  research.  Within  each  of  the  realms, 
technologies  and  policies  that  enable  condition  based  maintenance  are  discussed  with  a 
focus  on  cost  and  system  availability.  The  topics  covered  herein  form  the  foundation  for 
the  development  of  an  architecture  for  analyzing  the  effects  of  ISHM  on  CBM  and  the 
supply  chain.  Additionally,  chapters  4-7  contain  background  and  motivation  specific  to 
the  topics  covered  therein. 

Health  Monitoring  Concepts 

Structural  health  monitoring  (SHM),  while  sometimes  applied  as  monitoring  for  a 
system  at  large,  is  a  set  of  techniques  used  either  individually  or  in  conjunction  with 
others  to  determine  the  status  of  structural  components  of  a  vehicle  or  system.  SHM 
systems  use  a  variety  of  techniques  to  detect  the  onset  or  growth  of  damage,  i.e.,  cracks 
or  delaminations,  prior  to  failure.  Other  health  monitoring  sensing  systems  exist  to 
monitor  electrical  or  electronic  systems  as  well  as  rotating  machinery  (aircraft  engines)  or 
hydraulic  systems.  A  good  review  of  SHM  technology  and  application  is  found  in 
(Glaser,  Li,  Wang,  Ou,  &  Lynch,  2007).  Integrated  vehicle  health  management  (IVHM) 
and  integrated  systems  health  management  (ISHM),  used  interchangeably  in  published 
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works,  seek  to  compile  the  various  SHM  and  other  sensor  networks  and  interpret  the 
health  of  the  vehicle  or  system  as  a  whole.  This  processing  can  be  done  on  or  off-board 
the  system  in  question  depending  on  system  intent  and  requirements. 

Prognostics  and  health  management  (PHM)  is  synonymous  with  IVHM  and 
ISHM  in  most  literature,  emphasizing  the  prognostics  capability  to  detennine  the 
remaining  useful  life  (RUL)  of  a  component.  RUL  is  then  used  to  aid  in  the  determination 
of  when  and  how  to  maintain  the  system.  Examining  the  RUL  of  individual  components 
or  subsystems  can  assist  in  maintenance  planning  and  in  supply  requisitions,  transitioning 
from  time-based  replacement  to  repair  or  replacement  based  on  actual  material  condition. 
The  use  of  RUL  information  for  mission  planning,  maintenance  optimization  and  supply 
chain  management  is  often  referred  to  as  autonomic  logistics  (AL)  as  the  ISHM  system 
performs  these  functions  with  little  or  no  human  interaction.  Progressing  from  the 
component  level  sensing  systems  through  the  compilation  of  their  data  and  forecasting  of 
impending  failure  enables  the  transfonnation  in  maintenance  concepts.  Hess  (2005) 
decomposes  prognostics  and  health  management  (PHM)  with  three  terms: 

•  Enhanced  Diagnostics:  the  process  of  determining  the  state  of  a  component 
to  perform  its  function(s),  high  degree  of fault  detection  and  fault  isolation 
capability  with  very  low  false  alarm  rate; 

•  Prognostics:  actual  material  condition  assessment  which  includes  predicting 
and  determining  the  useful  life  and  performance  life  remaining  of  components 
by  modeling  fault  progression. 

•  Health  Management:  the  capability  to  make  intelligent,  informed, 
appropriate  decisions  about  maintenance  and  logistics  actions  based  on 
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diagnostics/prognostics  information ,  available  resources  and  operational 
demand,  (p.  3) 

These  three  terms,  while  broad,  form  the  foundation  for  measuring  the 
effectiveness  of  an  ISHM/CBM  system.  Enhanced  diagnostics  encompasses  probability 
of  detection  as  well  as  false  alarms,  directly  impacting  how  often  the  aircraft  must  be 
taken  out  of  service  for  inspection  and/or  repair.  Prognostics  are  the  ability  of  the  system 
to  forecast  the  remaining  useful  life  of  the  aircraft,  safety  of  flight  and  prediction  of 
impending  failures.  Health  management  takes  the  data  from  the  previous  two  and 
determines  when  to  order  spares  and  when  to  perform  maintenance. 

Maintenance  Concepts 

Maintenance  has  evolved  over  time  from  a  “fix  it  when  it  breaks”  policy  to  the 
current  focus  on  condition  based  maintenance  programs.  Table  1  presents  a  categorical 
breakdown  of  maintenance  approaches  and  their  attributes.  Rising  costs,  longer  service 
lives  and  reduced  manpower  have  driven  a  proactive  approach  to  maintaining  systems. 
The  reactive  or  corrective  maintenance  approach  forces  either  a  costly  spares  stockpile  to 
prepare  for  all  possible  failures  or  waiting  for  replacement  parts  to  arrive  resulting  in 
lower  operational  availability  (Amari,  McLaughlin,  &  Pham,  2006).  One  of  the  first 
advancements  in  maintenance  practice  was  to  establish  regular  inspection  and  preventive 
maintenance  (PM)  intervals.  These  time -based  techniques  analyze  failure  data,  either 
anecdotal  or  service  history,  to  determine  appropriate  timelines  to  inspect  and  replace 
components  or  systems  (Walls,  Thomas,  &  Brady,  1999).  This  PM  approach  results  in 
reduced  catastrophic  failures  and  more  predicted  maintenance  cycles  (Deputy  Under 
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Secretary  of  Defense  for  Logistics  and  Materiel  Readiness,  May,  2008).  Unanticipated 
failures  still  occur  outside  the  preprogrammed  maintenance  windows  and  must  be  taken 
into  account. 

Additionally,  PM  subjects  the  system  to  unnecessary  “repair”  based  on  the 
required  schedule  for  the  system.  The  unneeded  repair  adds  extra  expense  to  the  system 
since  the  component  had  remaining  useful  life,  and  it  increases  the  probability  of  damage 
resulting  from  the  maintenance  action.  The  diagnostic  CBM  approach  is  based  on 
inspections,  visual,  automatic,  non-destructive,  and  the  like,  and  parts  are  then  repaired  or 
replaced  as  required.  The  prognostic  CBM  approach  goes  a  step  further  than  the 
diagnostic  approach  and  takes  the  data  from  the  inspections  and  forecasts  when  repairs  or 
replacements  are  required.  As  Iyoob,  Cassady,  and  Pohl  (2006)  point  out,  a  majority  of 
studies  related  to  maintenance  practices  ignore  limited  budget,  manpower  or  time 
constraints.  They  present  logic  for  determining  maintenance  actions  when  the  time 
between  missions  does  not  provide  the  opportunity  to  make  all  desired  repairs. 

Condition  based  maintenance  and  other  predictive  maintenance  programs  have 
further  evolved  from  preventive  strategies  to  analyzing  system  health  and  forecasting 
remaining  life.  Time-based  inspections  alone  will  not  detect  all  failures,  especially  if  the 
inspection  timing  associated  with  the  component  is  such  that  a  critical  defect  occurs  at  an 
unanticipated  interval.  The  location  of  the  defect  also  plays  an  important  role  in  the 
failure  detection.  If  the  defect  is  located  in  an  infrequently  inspected  or  difficult  to  access 
area,  a  small  amount  of  damage  can  grow  to  catastrophic  amounts  prior  to  detection.  In 
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such  instances,  the  appropriate  application  of  SHM  technology  can  potentially  save  a 
system  from  severe  or  catastrophic  damage. 


Table  1.  Range  of  Maintenance  Approaches 


Maintenance  Approaches 


Category 

Reactive 

Proactive 

Sub- 

Category 

Run-to-fail 

Preventive 

Predictive 

Fix  when  it 
breaks 

Scheduled  maintenance 

Condition-based 

maintenance- 

diagnostic 

Condition-based 

maintenance- 

prognostic 

When 

Scheduled 

No 

scheduled 

maintenance 

Maintenance  based  on  a 
fixed  time  schedule  for 
inspect,  repair  and  overhaul 

Maintenance 
based  on  current 

condition 

Maintenance  based  on 
forecast  of  remaining 
equipment  life 

Why 

Scheduled 

N/A 

Intolerable  failure  effect 
and  it  is  possible  to  prevent 
the  failure  effect  through  a 
scheduled  overhaul  or 
replacement 

Maintenance 
scheduled  based 
on  evidence  of 
need 

Maintenance  need  is 
projected  as  probable 
within  mission  time 

How 

Scheduled 

N/A 

Based  on  the  useful  life  of 
the  component  forecasted 
during  design  and  updated 
through  experience 

Continuous 
collection  of 
condition 
monitoring  data 

Forecasting  of 
remaining  equipment 
life  based  on  actual 
stress  loading 

Kind  of 

Prediction 

None 

None 

On-  and  off- 
system,  near- 
real-time  trend 
analysis 

On-  and  off-system, 
real-time  trend 
analysis 

(Deputy  Under  Secretary  of  Defense  for  Logistics  and  Materiel  Readiness,  May,  2008) 


Why  CBM? 

Condition  based  maintenance,  when  appropriately  applied,  can  reduce  the  life 
cycle  cost  of  a  system.  The  United  States  Department  of  Defense  has  implemented  a 
program,  known  as  condition  based  maintenance  plus  (CBM+),  to  encourage  the 
implementation  of  processes  in  support  of  CBM  across  the  DoD.  Department  of  Defense 
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Instruction  (DoDI)  4151.22  defines  CBM+  as  “the  application  and  integration  of 
appropriate  processes,  technologies,  and  knowledge-based  capabilities  to  achieve  the 
target  availability,  reliability,  and  operation  and  support  costs  of  DoD  systems  and 
components  across  their  life  cycle”  (Under  Secretary  of  Defense  (AT&L),  May  2008,  p. 
1-1).  CBM  is  a  demand  driven  maintenance  process  based  on  indications  of  stress  or 
impending  failure  of  a  component  or  system.  Ellis  (2008)  argues  that  cost-effective 
systems  monitoring  allows  repair  actions  based  on  system  condition  rather  than  costly 
time-based  maintenance.  The  health  monitoring  system  monitors  component  sensor  data 
until  a  predetennined  point  prior  to  failure,  then  triggers  maintenance  to  repair  or  replace 
the  part.  CBM  can  also  proactively  examine  other  systems  in  the  vehicle  or  structure  as 
well  and  compile  this  data  with  projected  use  to  determine  the  maintenance  window  that 
optimizes  downtime,  manpower  and  spares.  Additionally,  interim,  time-based  inspections 
required  under  the  baseline  PM  approach  are  forgone  in  lieu  of  continuous  analysis  of  the 
aircraft  via  the  ISHM  system  (Ellis,  2008). 

CBM  Life  Cycle  Cost  Impacts 

The  real  challenge  is  to  develop  valid  and  measurable  metrics  for  quantifying  the 
impact  of  the  various  prognostic  technologies.  The  first  step  is  to  process  a 
cost/benefit  analysis  using  appropriate  modeling  tools,  for  each  component  and/ 
or  subsystem/system  to  evaluate  the  consequences  of  developing  and  supporting 
each  component  (Hess,  Calvello,  Frith,  Engel,  &  Hoitsma,  2006,  p.  6). 

Published  literature  shows  the  cost-saving  potential  of  condition  based 
maintenance  and  health  monitoring  systems  across  the  life  cycle: 
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•  40%  for  vehicle  maintenance  (Walls,  Thomas,  &  Brady,  1999) 

•  30%  to  50%  for  fuselage  panels  (Pattabhiraman,  Kim,  &  Haftka,  2010) 

•  10%  electrical  components  (Scanff  et  ah,  2007) 

•  50-80%  for  the  Boeing  777  (Gorinevsky,  Gordon,  Beard,  Kumar,  &  Chang, 

2005). 

The  issue  in  many  of  these  studies  is  the  failure  to  capture  costs  outside  of  a  few 
components  and  generally  restrictive  assumptions.  Additionally,  most  do  not  address  how 
the  prognostics  of  ISHM  can  impact  supply  timing  and  costs.  Since  maintenance  can  be 
forecast  to  match  impending  failures,  personnel  hours  can  be  managed  accordingly  to 
meet  demand. 

Several  authors  have  researched  and  proposed  methods  of  considering  life  cycle 
costs  (LCC)  for  systems,  and  they  discuss  methods  for  conducting  cost/benefit  analyses 
on  diverse  systems  to  determine  the  impact  of  an  ISHM  system  on  the  O&M  cost  and 
LCC  of  a  system,  but  fail  to  include  logistics  in  their  analyses.  Brand  and  Boiler  (2000) 
use  inspection  times,  research  and  development,  and  labor  costs  for  higher  level  cost  and 
time  savings  to  explore  inspections  on  commercial  aircraft.  Walls,  Thomas  and  Brady 
(1999)  examine  a  single  critical  component  of  hydraulic  servos  utilizing  a  decision  tree  to 
assess  probability  of  failures  and  MTBF  of  the  components,  neglecting  sensor  inputs. 
Lugtigheid,  Banjevic,  and  Jardine  (2005)  discuss  repair  or  replace  decisions  with 
imperfect  repair,  but  fail  to  use  prognostics  to  inform  their  repair  decisions,  instead 
utilizing  preventive  maintenance  approaches.  Banks,  Reichard,  Crow  and  Nickell  (2005) 
explore  battery  systems  for  armored  vehicles  using  prognostics  to  calculate  savings 
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through  a  return  on  investment  based  on  replacement  decisions.  They  discuss  logistics 
inputs,  but  do  not  include  them  in  the  model. 

Amari,  McLauchlin,  and  Pham  (2006)  research  CBM  with  manual  inspections  at 
predetennined  intervals  and  the  effect  of  use  conditions  on  the  wear  rate  of  components, 
utilizing  a  Markov  process  to  study  cost  savings.  Pattabhiraman,  Kim  and  Haftka  (2010) 
compare  manual  versus  ISHM  based  inspections  utilizing  maintenance,  manufacturing 
and  fuel  costs  for  an  aircraft.  Further  Pattabhiraman,  et  al,  discuss  a  hybrid  model  of 
manual  and  ISHM  inspections  on  a  system.  Gyekenyesi  (2013)  discusses  a  transitional 
approach  for  legacy  systems  not  previously  fitted  with  health  monitoring  systems  and  for 
systems  with  inadequate  coverage  by  using  non-destructive  inspection  (NDI).  He  further 
states  that  NDI  can  provide  confirmation  or  added  peace  of  mind  for  ISHM  system 
results  while  the  users  of  health  monitoring  systems  gain  trust  in  the  prognostics. 
Pattabhiraman  et  al.,  (2010)  also  discuss  a  hybrid  model  for  critical  items  which  could  be 
applied  in  a  similar  manner  as  Gyekenyesi.  While  these  methods  should  produce  savings, 
they  are  not  addressed  in  this  research. 

Wilmering  and  Ramesh  (2005)  assessed  the  impact  of  ISHM  approaches  on 
overall  system  ownership  costs.  They  argue  for  a  system  engineering  approach  to  assess 
and  blend  requirements  with  technologies  through  the  life  cycle  of  a  system  to  reduce 
cost.  A  seven  step  method  is  used  to  rank  ISHM  solutions  on  the  basis  of  component 
criticality,  failure  modes,  sensor  technologies  and  ISHM  approach  to  perfonn  a 
cost/benefit  analysis.  This  method  is  similar  to  the  failure  mode,  effects,  and  criticality 
analysis  (FMECA)  outlined  by  Blanchard  and  Fabrycky  in  Systems  Engineering  and 
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Analysis,  (2005).  The  aforementioned  process  is  conducted  within  The  Boeing  Company 
Ownership  Cost  Calculator  for  Aerospace  health  Management  (OCCAM)  tool.  The 
OCCAM  tool  captures  maintenance  and  logistics  processes,  both  recurring  and  non¬ 
recurring  costs.  A  key  item  examined  is  what  happens  when  a  part  fails,  specifically  the 
impact  on  the  maintenance  and  logistics  process.  Labor  rates,  material  costs,  inflation  and 
discount  rates  are  combined  to  determine  the  cost  impact  of  decisions.  Two  limiting 
assumptions  in  this  model  are  that  a  fixed  spares  demand  is  assumed  rather  than 
component  utilization  projected  by  the  health  management  system,  and  repairs  are 
perfect.  The  tool  also  provides  for  some  maintenance  scheduling  based  in  part  on  LCC; 
this  neglects  the  impact  on  availability  of  the  aircraft  which  can  play  a  large  role. 

Modares  and  Waksmanski  (2012)  state  early  detection  of  faults  with  health 
monitoring  can  limit  repair  costs  and  catastrophic  failures.  In  offshore  wind  turbines, 

SHM  tools,  maintenance  scheduling,  and  performance  of  the  SHM  system  determine  the 
added  value  of  the  system  of  systems  (A.  Van  Horenbeek,  Van  Ostaeyen,  Duflou,  & 
Pintelon,  2013).  Van  Horenbeek  et  al.  (2013)  lay  out  a  system  of  equations  that  includes 
as  inputs:  failure  cost;  component  reliability;  maintenance  costs;  and  spare  parts  costs,  to 
determine  the  added  value  of  health  monitoring  for  a  system.  This  system  provides  a 
thorough  examination  for  a  single  system  with  multiple  subcomponents.  It  does  lack, 
however,  inventory  management  and  the  effects  of  multiple  systems  operating 
simultaneously. 

The  cost  of  the  ISHM/CBM  system  must  be  accounted  for  in  the  LCC  evaluation 
as  well.  While  the  long  term  O&M  costs  may  be  significantly  reduced,  the  upfront  cost  of 
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development  and  deployment  of  the  systems  is  not  inconsequential.  With  the 
development  cost  for  the  F-35’s  ALIS  system  approaching  half  a  billion  dollars,  the 
technology  and  training  costs  play  a  role  (Butler,  2013).  Degradation  of  the  ISHM/CBM 
system  must  be  taken  into  account  as  well  as  degradation  in  the  overall  system,  in  this 
case  aircraft  performance  (e.g.,  increased  weight  and  power  requirements). 

Not  germane  to  the  LCC  methods  discussed  above  is  a  cost  benefit  analysis  and 
the  translation  from  private,  for  profit  frameworks,  to  military  applications.  Assigning 
cost  impacts  of  downtime  for  a  commercial  aircraft  is  different  than  for  a  military 
aircraft.  While  maintenance  and  spare  parts  costs  are  counted  in  both,  the  cost  of  not 
having  the  plane  in  operation  is  quantifiable,  at  least  on  a  broad  scale,  in  the  commercial 
sector.  Lost  revenue,  penalties  for  occupying  gates  too  long,  transferring  another  aircraft 
to  accommodate  passengers,  the  cost  of  refunding  or  transferring  passenger  tickets  can  all 
be  valued.  On  the  military  side,  attempting  to  value  downtime  and  ultimately  operational 
availability  is  a  much  more  difficult  task.  A  thorough  examination  of  the  system 
capabilities,  costs  and  infrastructure  paired  with  the  logistics,  operating  and  maintenance 
requirements  must  be  accomplished  to  accurately  assess  the  viability  of  an  ISHM  system 
to  reduce  the  cost  of  a  CBM  program. 

By  the  DoD’s  own  admission,  the  accuracy  of  current  LCC  figures  is  a  rough 
estimate  as  credible  means  to  measure  it  are  not  readily  available  (Under  Secretary  of 
Defense  (AT&L),  May  2008).  In  an  attempt  to  gain  a  clearer  picture  of  the  true  cost  of  a 
system,  the  DoD  also  proposes  a  Total  Life  Cycle  Systems  Management  (TLCSM) 
metric  of  cost  per  unit  of  operation  as  what  it  calls  “the  best  measure  of  life-cycle  costs,” 
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(Under  Secretary  of  Defense  (AT&L),  May  2008).  Ryan’s  assessment  of  the  DoD  LCC 
cost  accuracy  echoes  those  above,  “[djespite  the  fact  that  DoD  cost  estimating  practices 
have  become  increasingly  sophisticated,  the  actual  program  cost  estimates  that  are 
produced  remain  poor,  at  least  when  compared  to  the  final,  actual  costs  of  the  program” 
(Ryan,  Schubert,  Jacques,  &  Ritschel,  2013,  p.  73).  Capturing  these  costs  in  the 
respective  baseline  and  ISHM  models  will  likely  also  be  difficult.  Millar  (2007)  cites  that 
the  market’s  “invisible  hand”  holds  back  implementation  and  that  systems  engineering 
has  thus  far  not  demonstrated  the  case  for  use  of  ISHM  to  support  system  sustainment. 
The  “invisible  hand”  referred  to  includes  DoD  and  USG  policies  that  direct  the  Air  Force 
practices  in  acquisition,  maintenance  and  supply.  Additionally,  while  the  goal  is  to  save 
the  taxpayers  money,  the  DoD  does  not  have  the  commercial  profit  requirement  and 
pressure  found  in  industry. 

Maintenance  Activity  Grouping 

The  goals  of  grouping  maintenance  activities  utilizing  prognostics  are  reduction  in 
cost  to  maintain  and  increased  operational  time.  The  cost  savings  in  grouping 
maintenance  activities  comes,  in  general,  from  a  reduction  in  set-up,  tear-down  and 
reassembly  times,  yielding  a  reduction  in  labor  costs  for  multiple  individual  repair 
actions.  Operational  availability  is  increased  likewise  as  the  total  downtime  required  for 
these  repairs  or  replacements  is  reduced.  The  earlier  a  maintenance  action  is  scheduled, 
the  more  useful  life  is  wasted  but,  in  general,  the  probability  of  failure  is  lower. 
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Prognostics,  enabled  through  onboard  sensor  systems  allow  the  useful  life  to  be 
consumed  up  to  a  predetermined  risk  level  based  on  uncertainties  in  the  system. 

Camci  (2009)  utilizes  a  genetic  algorithm  approach  to  optimize  maintenance  for 
the  CBM  case  but  does  not  examine  the  benefits  over  a  PM  system.  The  approach  shows 
improvement  in  the  generic  cost  of  optimized  maintenance  using  prognostic  versus 
prognostics  enabled  CBM  alone.  Van  Horenbeek  and  Pintelon  (2013a;  2013b)  use  a 
heuristic  search  algorithm  to  detennine  the  optimal  maintenance  time  and  group  with 
results  showing  prognostic  maintenance  (CBM)  superior  to  standard  time -based 
maintenance  policy.  The  approaches  used  by  Camci  and  Van  Horenbeek  represent  most 
of  the  work  in  this  area,  but  do  not  address  the  impact  of  multiple  systems  with  multiple 
components  and  only  seek  to  optimize  for  cost  with  little  notion  of  the  impact  to  system 
operational  availability. 

Remaining  Useful  Life 

Since  achieving  the  theoretical  limit  of  100%  failure  avoidance  is  both 
impractical  and  wasteful,  we  are  forced  to  accept  some  level  of  risk.  (Hess  et  ah, 
2006,  p.  9) 

While  the  methods  and  techniques  used  for  remaining  useful  life  (RUL) 
calculations  are  outside  the  research  proposed  here,  their  use  is  essential  for  the  CBM 
maintenance  and  supply  processes.  A  good  summary  of  RUL  techniques  is  found  in  (Si, 
Wang,  Hu,  &  Zhou,  2011).  Risk  posture  and  acceptance  established  by  policy  or  unique 
mission  needs  help  to  determine  when  the  CBM  system  calls  for  replacement  of  a  given 
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component  or  system.  This  understanding  is  necessary  because  RUL  calculations  have 
inherent  uncertainty  as  they  are  generally  derived  based  upon  probability  distributions. 
After  probability  of  detection,  which  is  a  function  of  the  sensor  type  and  software, 
remaining  useful  life  prognostics  are  the  most  vital  processes  performed  by  the  ISHM 
portion  of  the  ISHM/CBM  system. 

RUL  inference  must  take  into  account  the  P-F  curve  in  Figure  2  for  individual 
components,  where  P  is  the  potential  failure  or  point  at  which  deterioration  begins  or  is 
detected.  The  F  is  the  functional  failure  or  where  the  part  or  system  actually  breaks.  The 
horizontal  lines  on  the  curve  correspond  to  limits  that  are  either  functions  of  the  system 
or  prescribed  by  policy  or  directives.  System  detection  limit  is  a  function  of  the  ISHM 
system  and  physical  attributes  of  the  aircraft;  the  point  of  detection  (B)  occurs  where  the 
failure  curve  of  the  component  crosses  the  detection  limit.  Notification,  a  policy  driven 
component,  occurs  at  point  C  and  begins  the  CBM  supply  and  maintenance  planning 
window.  For  components  with  safety  limits,  i.e.,  those  that  would  cause  a  catastrophic 
loss,  point  D  is  where  the  system  would  be  taken  down  for  maintenance. 
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Figure  2.  Remaining  Useful  Life  Schematic  (Jennions,  2011) 

In  the  case  of  a  time  or  prediction  based  “failure”  when  the  part  is  replaced  with 
some  serviceable  life  remaining,  policy  based  on  observed  and  physics  based  estimates  of 
failures  dictates  the  appropriate  time  frame.  The  goal  is  that  the  probabilistic  estimates  of 
RUL  based  on  real  time  monitoring  allow  increased  time  to  accumulate  on  parts,  thus 
increasing  the  MTBF  for  the  ISHM  aircraft  and  total  costs  associated  with  replacement 
parts. 


Supply  Chain  Forecasting  for  ISHM  Aided  CBM  Systems 

ISHM  systems  in  the  US  Air  Force  are  currently  limited  to  the  F-22,  which 
requires  manual  collection  and  analysis  of  sensor  data,  and  test  articles  on  individual 
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systems  and  limited  engine  monitoring  systems  on  several  other  platfonns.  These  efforts 
are  not  integrated  with  the  logistics  chain  to  optimize  maintenance  actions  with  supply 
requirements.  While  individual  systems  addressing  health  monitoring  exist  in  the  USAF 
inventory,  e.g.,  F-22  and  C-17,  the  future  goal  of  Air  Force  ISHM/CBM  integration,  lies 
in  the  F-35.  One  of  the  F-35’s  keys  to  success  and  reduction  in  O&S  costs  is  the 
Autonomic  Logistics  Information  System  (ALIS). 

ALIS  was  designed  to  be  a  one-stop  system  to  monitor  aircraft  systems,  predict 
system  failures,  increase  maintenance  efficiency  and  streamline  the  supply  stream  by 
combining  the  range  of  systems  currently  in  use  for  existing  Air  Force  aircraft.  The  F-16, 
which  is  the  standard  for  legacy  aircraft,  has  separate  and  distinct  systems  for  supply, 
maintenance,  and  pre  and  post  mission  inspections  (Butler,  2013).  These  legacy  systems 
require  multiple  career  fields  and  administrative  processes  that  increase  the  turn  time 
between  aircraft  missions.  Communication  between  the  aircraft  and  ground  systems  by 
ALIS  allows  optimization  of  down-time  and  prepositioning  of  required  parts,  ensuring 
maintenance  personnel  maximize  the  availability  of  the  aircraft.  An  example  of 
maintenance  time  savings  is  the  use  of  electronic  tracking  of  fluid  levels  versus  manual 
gauges.  Additionally,  Butler  (2013)  states  that  flight  control  maintenance  on  the  F-35  is 
reduced  from  8-14  hours  on  legacy  aircraft  to  5  minutes.  Further  details  on  this  dramatic 
reduction  are  not  provided  in  the  article.  When  maintenance  actions  across  the  F-35  are 
compounded,  the  potential  for  large  downtime  savings  is  immense.  Whether  or  not  these 
savings  can  be  realized  is  yet  to  be  seen.  As  the  F-35  approaches  initial  and  then  full 
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operating  capability,  maintenance  and  logistics  data  will  be  analyzed  to  determine  the 
true  value  of  the  ALIS  system. 

Perhaps  the  key  feature  of  ALIS  is  the  prognostic  capability.  The  prognostics  and 
health  management  (PHM)  system  of  the  F-35  shifts  maintenance  policy  from  preventive 
time-driven  routines  to  condition  or  demand  driven  actions.  The  PHM  system  calculates 
the  remaining  useful  life  of  a  system  or  component  which  flows  into  the  supply  pipeline 
to  ensure  lead  times  for  part  orders  are  met  when  maintenance  is  required.  The  fielding  of 
ALIS  or  any  similar  component  or  vehicle-wide  system  on  legacy  aircraft  will  require 
concept  of  operations  (CONOPs)  changes  for  maintenance  and  operations.  These 
changes,  while  they  must  be  enforced  and  driven  from  higher  command  levels,  must 
obtain  “buy-in”  at  the  squadron  level  as  they  are  unproven  and  potentially  unreliable. 
Byer,  Hess,  and  Fila  (Byer,  Hess,  &  Fila,  2001)  project  that  PHM’s  automation  capability 
can  eliminate  bureaucracy  built  up  in  military  logistics  and  supply  systems.  If  operational 
change  can  be  affected,  streamlining  the  maintenance,  supply  and  operations  activities 
proposed  in  the  ALIS  system  are  achievable. 

The  ALIS  system  is  not  without  problems  though,  along  with  the  rest  of  the  F-35 
program.  With  a  development  cost  approaching  $448  million,  ALIS  is  not  yet  proven  and 
has  security  flaws  (Butler,  2013).  Additionally,  the  system  is  behind  schedule  and  fails  to 
meet  maintenance  and  sortie  generation  requirements  for  the  F-35.  “To  date,  diagnostic 
system  performance  has  failed  to  meet  basic  functional  requirements,  including  fault 
detection,  fault  isolation,  and  false  alarm  rates,”  (Director,  Operational  Test  & 

Evaluation,  2013,  p.  49).  In  the  interim,  manual  intervention  by  maintainers,  using  legacy 
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aircraft  techniques,  and  contractor  support  are  used  to  keep  the  aircraft  flying  thus 
negating  the  cost  savings  that  could  be  afforded  by  the  ALIS  system. 

Logistics  Methods 

Supply  chain  optimization  is  handled  in  many  ways  across  industry.  Some  smaller 
businesses  manufacture  all  required  parts  in  house  and  fix  them  as  needed.  This  is  seldom 
the  case  though  and  most  businesses  and  industries  depend  on  collaboration  with  other 
agencies. 

In  the  commercial  world,  profit  is  the  end  goal  for  companies  and  should  be  the 
selling  point  for  ISHM  systems.  As  Grubic  (2009)  points  out,  PHM  can  shift  the 
operating  paradigm  for  a  system  but  capturing  the  impact  to  potential  increases  in 
revenue  is  often  overlooked  aside  from  the  often  cited  reduction  in  maintenance  costs. 

The  ability  to  foresee  maintenance  issues  and  optimize  when  systems  are  taken  offline  for 
repair  is  a  key  part  of  this  revenue  generation.  Unplanned  downtime  drives  millions  of 
dollars  in  cost  for  organizations  with  unplanned  maintenance  costing  three  to  ten  times 
that  of  scheduled  or  predicted  maintenance  activities  (Taft,  2013).  This  is  a  key 
difference  between  military  and  commercial  activities  in  the  logistics  and  maintenance 
area.  Commercial  companies  are  in  business  to  make  a  profit,  whereas  military  forces 
face  no  such  requirement  other  than  to  be  good  stewards  of  taxpayer  dollars. 

In  the  commercial  aerospace  field,  Boeing  is  an  industry  leader  in  analyzing 
logistics  requirements  for  aircraft  fleets.  Boeing’s  Airplane  Health  Management  (AHM) 
system  takes  real-time  data  from  the  aircraft  to  identify  and  diagnose  faults.  This  data  is 
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used  to  provide  error  identification  and  historical  tracking  to  aide  in  fault  troubleshooting 
to  optimize  operations  and  maintenance  efficiencies  (The  Boeing  Corp,  2013). 
Additionally,  Boeing  developed  an  Arena8  discrete  event  simulation  model  for  the  civil 
aviation  sector  to  analyze  the  perfonnance  of  health  management  options.  (Williams, 
2006)  The  AHM  system  performs  similar  functions  as  the  ALIS  system,  with  the 
commercial  addition  of  providing  a  decision  tool  to  conduct  critical  profit  analyses.  As 
shown  with  both  the  ALIS  and  AHM  systems,  prognostics  integrated  with  maintenance 
and  supply  chains  can  enable  condition  based  maintenance.  It  is  this  end  state  that  the  Air 
Force  and  DoD  as  a  whole  is  striving  to  meet,  not  to  turn  a  profit  as  in  private  industry, 
but  to  continue  operations  in  a  shrinking  fiscal  environment. 

Air  Force  Supply  Issues 

The  General  Accountability  Office  conducted  several  audits  of  DoD  and  in 
particular  Air  Force  supply  systems  over  the  past  20  years.  Their  findings  range  from 
lack  of  items  to  enormous  on-hand  stock  and  unneeded  parts  on  order  accounting  for 
billions  of  dollars  in  unneeded  parts  and  operational  rates  well  below  goals.  (Government 
Accountability  Office,  2001;  Government  Accountability  Office,  2007)  “Having  spare 
parts  available  when  needed  to  perfonn  required  maintenance  is  critical  to  the 
Department  of  Defense’s  accomplishment  of  its  missions.  Shortages  of  spare  parts  are  a 
key  indicator  of  whether  the  billions  of  dollars  annually  spent  on  these  parts  are  being 
used  in  an  effective,  efficient,  and  economical  manner,”  (Government  Accountability 
Office,  2001,  p.  1).  “In  January  2001,  we  reported  on  Department  of  Defense 


28 


management  challenges  and  noted  that  the  Department  has  had  serious  weaknesses  in  its 
management  of  logistics  functions  and,  in  particular,  inventory  management,” 
(Government  Accountability  Office,  2001,  p.  3).  These  GAO  reports  along  with  several 
DoD  initiatives  to  correct  the  findings  are  discussed  in  this  section. 

The  spare  parts  shortage  leads  maintainers  to  cannibalize  parts,  leading  to  extra 
work  to  fix  the  aircraft  they  “borrowed”  from.  This  cannibalization  opens  the  possibility 
of  damaging  the  needed  part  in  removing  it  from  the  donor  system  as  well  as  causing 
collateral  damage  in  the  process.  The  “borrowed”  part  will  also  generally  not  last  as  long 
as  a  new  component,  thus  requiring  additional  maintenance  (Government  Accountability 
Office,  2001).  Reasons  for  parts  shortages  range  from  inadequate  forecasting,  repair  and 
manufacturing  issues,  poor  replacement  part  reliability,  and  contracting  problems.  In  one 
extreme  example,  demand  for  an  engine  bolt  was  projected  at  828  units  when  actual 
requirements  were  over  12,000  in  a  single  quarter.  (Government  Accountability  Office, 
2001)  While  this  is  an  extreme  example,  it  is  indicative  of  problems  with  the  forecasting 
models  used  by  the  Air  Force  to  detennine  part  requirements.  This  issue  was  still  present 
in  a  2010  review  of  DoD  logistics  citing  poor  requirements  projections  as  a  leading  cause 
of  inventory  shortages  and  surpluses  (Atchley  et  ah,  2010).  Implementing  ISHM/CBM 
systems  on  a  broad  range  across  the  service  would  increase  the  forecasting  capability  and 
provide  near  real  time  insights  into  requirements,  both  existing  and  in  the  near  future. 
Between  2002  and  2005  Air  Force  spares  shortages  were,  as  seen  in  the  Table  2  below, 
between  6  and  8%  of  total  requirements  averaging  over  $  1  billion  is  shortfalls.  The 
reasons  given  then  were  the  same  as  in  2001  and  again  in  2010. 
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Table  2.  Air  Force  Inventory  Shortages  from  FY  2002  through  FY  2005 


Dollars  In  billions 


Shortages 


Fiscal  year 

Value  of  total  requirements 

Number  of  items 

Value 

Percent 

2005 

S15.8 

371.961 

SI. 2 

7.6% 

2004 

15.8 

691,509 

1.1 

7.0 

2003 

14.6 

356.977 

0.9 

6.2 

2002 

14.4 

428.195 

1 2 

8.3 

(Government  Accountability  Office,  2007) 


A  shortage  of  spares  is  not  the  only  issue  facing  the  Air  Force.  The  GAO  (2007) 
stated  that  Air  Force  on-order  spares  without  projected  need  for  future  use  accounted  for 
$1.3  billion  or  52%  of  on-order  parts.  DoD  and  Air  Force  policy  do  not  incentivize  or  in 
some  cases  allow  for  cancellation  of  orders  without  significant  monetary  penalties. 
Additionally,  between  2002  and  2005  65%  of  on-hand  inventory,  amounting  to  $18.7 
billion,  was  not  required  for  projected  use  rates.  The  additional  inventory  was  calculated 
by  the  GAO  to  cost  another  $15  million  yearly  for  storage.  Further  compounding  the 
unneeded  on-hand  inventory  is  that  some  items  have  a  shelf  life  that  requires  disposal  or 
refurbishment  after  a  set  amount  of  time.  The  problem  does  not  lie  entirely  within  the  Air 
Force  supply  system  control.  The  DoD’s  procurement  policies  drive  sparing  decisions  to 
be  made  between  2  and  4  years  prior  to  actual  need  of  the  part  due  to  lead  times  and  the 
DoD  budgeting  process  (Atchley  et  ah,  2010).  This  is  a  military  issue  that  is  not  faced  on 
the  same  scale  in  the  corporate  arena.  A  2010  study  on  DoD  logistics  does  comment  on 
the  difficulty  of  forecasting  spares  requirements  stating  these  projections  are  rarely  100 
percent  accurate.  The  prognostics  capability  of  ISHM  systems  can  increase  the  reliability 
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of  spares  forecasting  and,  depending  on  projected  use  rates,  keep  sparing  purchases 
within  DoD  budget  timelines. 

Because  the  military  flies  its  aircraft  beyond  original  service  lives,  B-52  and  KC- 
135  for  example,  supply  and  maintenance  issues  exist.  On  one  hand,  if  repairs  can  be 
made  on  long  life  items,  it  can  save  money,  but  when  you  need  a  new  part  eventually,  the 
supplier  doesn’t  make  them  anymore.  An  interesting  dilemma  arises  when  there  is  only 
one  entity  driving  demand  for  a  system  such  as  a  military  aircraft.  Organic  military  repair 
capabilities  can  drive  the  demand  for  the  part  so  low  it  no  longer  makes  financial  sense 
for  the  manufacturer  to  support  smaller  components,  thus  when  the  part  can  no  longer  be 
repaired  it  cannot  be  purchased.  Efforts  to  maintain  industrial  base  through  prescribed 
purchases  can  keep  production  lines  open  but  do  little  to  address  over  supply  of  some 
items  in  stock.  Additionally,  there  is  a  demand  driven  issue  for  military  hardware  during 
times  of  conflict.  Post-September  11,  2001,  Air  Force  aircraft  experienced  a  marked 
increase  in  sortie  rates  and  flying  hours,  leaving  logistics  projections  ineffective  in 
determining  requirements  (McCoy,  2011).  While  an  ISHM/CBM  system  would  do  little 
to  project  a  wartime  surge,  it  could  provide  a  fleet-wide  look  at  the  remaining  useful  life 
of  parts  and  allow  decisions  on  purchasing  priorities  to  be  made.  The  ability  to  extend  the 
life  on  systems  through  the  prognostic  capability  of  ISHM  also  allows  aircraft  to  operate 
closer  to  the  safe  life  limits  of  parts  as  real  time  condition  can  be  obtained.  The  goal  of 
ISHM  enabled  CBM  systems  is  matching  aircraft  requirements  with  the  supply  and 
maintenance  processes.  By  ensuring  parts  are  available  when  needed,  while  keeping  only 
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the  required  material  on  hand,  supply  and  storage  costs  are  reduced  while  maintenance 
downtime  can  be  managed. 

Modeling  Approaches 

Rebulanan  models  7  elements  in  his  JAVA  model  simulating  the  JSF  ALS 
system:  aircraft;  health  management  system  and  LRUs;  communication  system;  supply 
(depot,  base,  and  flightline)  and;  maintenance  (Rebulanan,  2000).  Rebulanan  uses  3 
MOPs  to  evaluate  model  performance.  Aircraft  availability,  both  the  number  of  aircraft 
per  day  and  percentage  available  for  mission  (A0),  average  number  of  sorties  per  day  and 
average  wait  time  for  supply.  Rebulanan’ s  model  shows  sensitivity  of  the  supply  wait 
time  to  the  PHM  detection  lead  time  for  an  impending  failure,  and  the  supply  stock 
levels.  This  outcome  is  somewhat  intuitive  in  that  as  the  prognosis  of  an  impending 
failure  is  detected  earlier  and  with  greater  accuracy,  the  supply  system  can  plan  further  in 
advance,  ensuring  parts  are  available  when  required.  Malley  (2001)  developed  a  JAVA 
model  based  on  Rebulanan’ s  work  with  a  focus  on  PHM  intricacies.  Malley  examines 
batching  to  make  RUL  predictions,  that  is,  processing  data  grouped  over  a  set  duration 
rather  than  instantaneously  to  dampen  noise  in  the  signal.  The  findings  showed  batch 
processing  decreased  the  false  alarm  rate  over  the  life  of  a  part,  but  delayed  detection 
time  of  impending  failures. 

Yager  (2003)  approached  modeling  autonomic  logistics  for  aircraft  as  a  queuing 
theory  model  to  examine  sortie  generation  rates.  The  model  shows  a  higher  sortie 
generation  rate  for  the  ISHM  aircraft.  Yager’s  model  does  have  some  drawbacks  for 
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implementation  on  a  larger  and  closer  to  realistic  system  in  that  there  is  no  penalty  for 
false  alarms  in  the  ISHM  system  and  prognosis  is  assumed  to  be  perfect. 

Rodrigues  and  Yoneyama  (2012;  2013)  explore  the  effect  of  prognostics  on  spare 
parts  inventories  for  both  repairable  and  non-repairable  systems  compared  with 
conventional  supply  processes.  Both  studies,  simulated  over  15  years  each,  show  cost 
savings  for  the  ISHM  enabled  system  over  the  conventional  one.  For  the  non-repairable 
model  a  discrete  event  simulation  is  used  and  considers  parts,  storage  and  out  of  stock 
costs.  Out  of  stock  costs  are  difficult  to  quantify  but  do  impact  downtime  for  supply, 
which  is  where  the  impact  is  captured  in  this  research  model.  The  supply  system  is 
represented  by  a  reorder  point  and  an  order  quantity  with  varying  levels  for  each.  In  the 
conventional  system  these  are  fixed,  but  in  the  ISHM  model  they  are  updated  based  upon 
system  inferences.  A  limitation  of  this  study  is  that  only  one  item  is  investigated,  leaving 
interactions  of  multiple  components  in  question. 

Exploring  the  effect  of  RUL  inferences  on  repairable  systems  Rodrigues  and 
Yoneyama  (2013)  found  an  improvement  in  fleet  availability  through  managing  when 
items  were  sent  for  repair.  Holding,  repair  and  out  of  stock  costs  were  used  again  as  the 
measure  for  cost.  Sparing  levels  were  varied  for  the  systems  and  costs  and  system 
availability  favored  the  ISHM  scenario  at  each  level.  This  model  does  not  include 
maintenance  costs  or  interactions  thus  the  system  availability  cannot  be  considered 
operational  availability. 
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Summary  of  Gaps 

The  review  of  current  literature  in  this  chapter  highlights  that  most  modeling 
efforts  lack  inclusion  of  logistics  in  their  research.  As  such,  the  models  cannot 
comprehensively  explore  the  interactions  between  ISHM,  CBM  and  logistics  systems. 
Further,  the  inclusion  of  multiple  systems  in  this  research,  a  squadron  of  aircraft  with 
multiple  components  is  left  unexplored.  This  gap  could  yield  additional  cost  and  time 
savings  if  the  health  monitoring,  maintenance  and  logistics  system  interactions  can  be 
synchronized.  Using  the  information  above  to  optimize  or  group  maintenance  activities 
while  maximizing  Ao  and  minimizing  cost  for  a  system  of  systems,  is  another  area  of 
research  open  for  study.  Further  approaches  and  their  associated  benefits  and  limitations 
are  discussed  in  chapters  4-7. 
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III.  System  Architecture  and  Metrics 


Methods  to  Evaluate  ISHM/CBM  Systems 

The  DoD  explains  Condition  Based  Maintenance  Plus  (CBM+)  as  the  use  of 
processes  and  technologies  to  increase  the  reliability  and  maintenance  effectiveness  of 
systems  (Deputy  Under  Secretary  of  Defense  for  Logistics  and  Materiel  Readiness,  May, 
2008).  Further,  maintenance  is  perfonned  based  on  need  utilizing  systems  engineering 
approaches  to  collect  and  analyze  data  in  support  of  the  decision-making  processes.  The 
DoD  Condition  Based  Maintenance+  Guidebook  prescribes  four  life-cycle  sustainment 
outcome  metrics  to  evaluate  CBM+  implementation  in  the  Total  Life  Cycle  Systems 
Management  (TLCSM)  process: 

Materiel  availability  (MA)  is  a  measure  of  the  percentage  of  the  total  inventory  of 
a  system  that  is  operationally  capable  (ready  for  tasking)  of  performing  an 
assigned  mission  at  a  given  time,  based  on  materiel  condition.  It  can  be  expressed 
mathematically  as  the  number  of  operational  end  items  [FMC]  divided  by  the 
total  population  [feet  size] .  Materiel  availability  also  indicates  the  percentage  of 
time  a  system  is  operationally  capable  of  performing  an  assigned  mission. 
Materiel  reliability  (MR)  is  a  measure  of  the  probability  the  system  will  perform 
without  failure  over  a  specific  interval.  Reliability  must  be  sufficient  to  support 
the  warfighting  capability  needed.  Materiel  reliability  is  generally  expressed  in 
terms  of  a  mean  time  between  failures  (MTBF),  and,  once  operational,  can  be 
measured  by  dividing  actual  operating  hours  by  the  number  of  failures 
experienced  during  a  specific  interval. 

Ownership  cost  (OC)  balances  the  sustainment  solution  by  ensuring  the  O&S 
costs  associated  with  materiel  readiness  are  considered  when  making  decisions. 
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Mean  downtime  (MDT)  is  the  average  total  time  required  to  restore  an  asset  to 
its  full  operational  capabilities.  MDT  includes  the  time  from...  an  asset  being 
down  to  the  asset  being  given  back  to  operations  or  production  to  operate. 
(Deputy  Under  Secretary  of  Defense  for  Logistics  and  Materiel  Readiness,  May, 
2008,  p.  1-5) 

The  above  metrics  align  with  the  Sustainment  key  perfonnance  parameter  (KPP) 
spelled  out  in  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS) 
manual  (2015).  Falling  under  the  availability  KPP,  MA  and  Ao  represent  operational 
capability.  Material  reliability  maps  to  the  JCIDS  reliability  KSA,  and  OC  falls  under  the 
O&S  cost  KSA.  The  combination  of  these  high  level  metrics  is  an  aggregation  of  the 
measures  of  effectiveness  (MOEs)  and  measures  of  performance  (MOPs)  for  the  baseline 
and  monitored  systems. 

Systems  Architecture  Approach 

The  foundation  of  an  architecture  for  a  system  is  the  ability  to  map  key  program 

objectives  to  system  performance  and  design.  Table  3  lists  the  interaction  between  key 

CBM+  objectives  and  the  aforementioned  metrics  where  MA  is  materiel  availability,  MR 

is  materiel  reliability,  OC  is  the  ownership  cost  and  MDT  is  the  mean  downtime.  The 

four  metrics  are  discussed  further  in  the  next  section.  These  objectives  are  not  all 

quantitatively  measurable,  but  must  still  be  considered  and  assessed  when  qualitatively 

evaluating  the  system.  In  mapping  these  objectives  to  the  metrics,  the  DoD  provides  a 

conduit  for  candidate  approaches  to  ensure  they  meet  the  goals  for  the  process.  The 

relationship  between  objectives  and  their  respective  metrics  and  subsequent 
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decomposition  into  system  level  measures  of  perfonnance  affords  the  opportunity  to 
compare  multiple  approaches  using  the  same  upper  level  metrics. 

Table  3.  CBM+  Objectives  and  Metrics 


OBJECTIVE 

MA 

MR 

OC 

MDT 

Enhance  maintenance  effectiveness  with  integrated 
maintenance  and  logistics  systems 

X 

X 

Incorporate  advanced  engineering,  maintenance, 
logistics/supply  chain,  configuration  management,  and 
infonnation  technologies 

X 

Employ  weapon  system  designs  that  use  measurable, 
consistent,  and  accurate  predictive  parameters  from 
embedded  CBM  capabilities 

X 

Improve  data  about  maintenance  operations  and  parts/system 
performance 

X 

X 

Improve  advanced  diagnostics,  system  prognostics,  and  health 
management  capabilities  based  on  current  condition  data 

X 

X 

X 

X 

Provide  more  accurate  item  tracking  capabilities 

X 

Reduce  maintenance  requirements  by  performing 
maintenance  tasks  only  upon  evidence  of  need  (more 
proactive/predictive,  less  preventive  and  less  corrective) 

X 

X 

Enable  more  effective  maintenance  training 

X 

Create  a  smaller  maintenance  and  logistic  footprint 

X 

Improve  maintenance  capabilities,  business  processes, 
supply/maintenance  planning,  and  responsiveness  leading  to 
optimum  weapon  system  availability 

X 

Minimize  unique  support  equipment  and  information 
systems  for  individual  weapon  systems 

X 

Improve  system  maintainability  as  a  part  of  design 
modification  through  the  use  of  reliability  analysis 

X 

X 

Provide  interoperability/jointness  to  the  warfighter 

X 

X 

(Under  Secretary  of  Defense  (AT&L),  May  2008) 


Detailed  cost  comparisons,  made  possible  by  comprehensive  system  architecture, 
along  with  maintainability  studies  provide  the  foundation  for  assessing  the  feasibility  of  a 
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SHM  system  to  compliment  CBM.  As  Grubic  (2009)  points  out,  many  approaches  to 
ISHM/CBM  “have  been  developed  by  engineers  for  engineers  and  therefore  suffer  from 
lack  of  business  input.  The  latter  brings  with  it  a  view  of  the  customer,  and  the  cultural 
and  process  drivers  that  are  so  important  to  the  success”  (p.  2).  Byer  lays  out  a  process  to 
evaluate  an  ISHM/CBM  system: 

1.  Define  a  Baseline  System  without  PHM  and  the  Aircraft  System  with  PHM; 

2.  Develop  Reliability  and  Main  tainability  Predictions  for  the  Componen  ts  of  the 

Aircraft; 

3.  Define  the  Measures  of  PHM  Effectiveness; 

4.  Metrics  Associated  with  the  Measures  of  Effectiveness; 

5.  Estimating  the  Impact  of  PHM  on  Reducing  the  Cost  of  Consumables; 

6.  Estimating  the  Impact  of  PHM  on  Reducing  Maintenance  manpower  Costs; 

7.  Estimating  the  Non-recurring  and  Recurring  Costs  of  Providing  PHM; 

8.  Develop  the  Cost  Benefit  Results; 

9.  Estimating  the  Impact  of  PHMs  on  Non  Dollar  Denominated  Benefits; 

a.  Sortie  Generation  Capability 

b.  Reduction  in  the  Incidence  of  Major  Accidents  (Byer  et  ah,  200 1 ,  p. 

3098). 

In  new  structures  and  vehicles,  including  both  CBM  and  ISHM  in  the  system 
architecture  early  in  the  process  allows  for  trade-offs  to  be  made  on  cost,  complexity  and 
operational  availability  (Ao).  For  fielded  systems,  the  complexity  increases  since  most 
systems  are  deployed  without  health  monitoring  allowances  and  must  be  retrofitted 
which,  depending  on  level  of  access,  can  carry  significant  cost  and  weight  penalties. 
Sensors  and  their  associated  cables  or  wireless  communication  system  and,  if  applicable, 
on-board  diagnostic  system  have  weight  and  power  requirements  also  which  can  have 
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significant  perfonnance  impacts  in  the  case  of  aircraft.  System  availability  is  linked  to  the 
balance  of  sensor  reliability  and  detectability  and  the  capability  of  the  system  to  decrease 
maintenance  duration  (Hoyle,  Mehr,  Turner,  &  Chen,  2007). 

As  stated  previously,  the  benefits  of  ISHM  are  the  abilities  to  reduce  inspection 
length,  defer  maintenance  and  migrate  to  maintenance  on  demand  or  CBM  (Speckmann, 
2007).  All  of  these  items  have  the  end  goal  to  increase  operational  availability  through 
reduced  maintenance  time.  The  link  between  the  CBM  program  and  the  SHM  technology 
is  system  architecture.  Within  sectors  of  industry,  there  are  attempts  to  define  standards 
or  common  operating  practices  and  to  persuade  governments  and  private  companies  to 
invest  in  SHM  technology  to  reduce  overall  maintenance  costs.  These  will  in  turn  enable 
measuring  the  effectiveness  and  cost  of  individual  systems. 

Applying  ISHM  enables  CBM  in  lieu  of  preprogrammed  periodic  maintenance 
practices,  that  is,  maintaining  only  when  required  instead  of  when  prescribed  by 
schedules,  thus  optimizing  maintenance  labor  (Roach,  2009).  All  of  the  SHM 
technologies  and  resulting  modified  maintenance  programs  serve  to  reduce  the  total 
acquisition  cost  of  a  system  with  increased  availability.  System  reliability  is  also  cited  as 
a  reason  for  the  lack  of  further  adoption  of  the  technology.  Seaver  et  al.  (2012)  state  that 
most  ISHM  systems  lack  demonstrated  reliability  in  the  field  beyond  estimates  which,  in 
turn,  discourages  program  managers  from  investing  in  the  technology  for  deployable 
systems.  To  gain  user  trust  in  predictions  ISHM  systems  should  compile  and  quantify 
uncertainty  to  provide  some  confidence  interval  around  predictions. 
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Processing  CBM  data  can  prove  cumbersome  as  well.  Depending  on  the  scope 
and  size  of  the  system,  the  ISHM  platfonn  can  produce  terabytes  of  data  every  month 
(Seaver  et  ah,  2012).  Methods  to  collect,  process,  analyze  and  store  this  data  must  be 
accounted  for  when  developing  and  assessing  an  ISHM  system  (Farrar  &  Worden,  2007). 
While  this  is  a  measure  to  track  and  be  aware  of  for  a  system,  modeling  this  falls  under 
final  design  for  the  system  and  will  not  be  included  in  this  model.  The  cost  associated 
with  the  design,  development,  implementation  and  maintenance  of  these  systems  must  be 
accounted  for  in  the  ISHM  model  in  order  to  provide  an  accurate  comparison  of  the  cost 
benefit  of  the  baseline  system  versus  the  ISHM/CBM  system. 

Model  Functionality 

Figure  3  lays  out  the  process  flows  and  functional  components  of  for  the  baseline, 
no  ISHM,  aircraft  through  the  models  in  this  research.  ISHM  architecture  is  presented  in 
chapter  4  and  inherent  ISHM  processes  are  discussed.  The  models  are  designed  to 
account  for  the  processes  and  decisions  in  the  appropriate  flows.  Following  this  process 
ensures  the  ability  to  capture  the  required  processing  times  and  failures  for  calculation  of 
system  impacts. 


40 


41 


In  the  nominal  baseline  model  in  Figure  3  the  aircraft  are  tasked  for  a  mission  and  then 
proceed  to  a  preflight  inspection,  mission  execution,  post  flight  inspection  and  routine 
maintenance  before  returning  to  the  mission  queue.  Off-ramps  can  occurs  if  inspections 
reveal  faults  that  require  correction  or  if  there  is  an  in  flight  emergency  (IFE).  Following 
the  identification  of  faults,  the  aircraft  receives  a  detailed  inspection  to  isolate  and  locate 
the  offending  component  or  line  replaceable  unit  (LRU)  for  repair  or  replacement,  a 
supply  requisition  is  generated  and  the  stock  level  is  checked.  If  the  part(s)  are  in  stock 
they  are  sent  to  maintenance  to  repair  the  aircraft.  If  the  required  items  are  not  in  stock, 
they  must  be  ordered  and  the  aircraft  is  down  awaiting  supply.  Upon  repair  of  the  aircraft, 
the  system  is  again  checked  for  damage.  If  no  damage  is  detected,  the  aircraft  is  sent  back 
into  service,  repeating  the  cycle  until  the  aircraft  is  retired  or  destroyed.  The  ISHM 
enabled  CBM  architecture  is  discussed  in  chapter  4. 

Data  Flow 

The  schematic  outlined  in  Figure  4  shows  the  information  that  is  transferred 
between  the  different  system  nodes  in  the  ISHM  model.  The  ISHM  system  updates  the 
CBM  system  with  systems  health  and  RULs.  This  information  is  then  processed  by  the 
CBM  system,  which  determines  maintenance  and  supply  requirements.  Supply 
requirements  are  sent  to  the  supply  system  and  parts  statuses  are  returned.  Likewise, 
maintenance  windows  and  the  subsystems  to  be  repaired  are  sent  from  the  CBM  system 
to  maintenance  and  maintenance  updates  the  CBM  system  on  status  of  repairs.  All  of  this 
information  is  tracked  to  ensure  enough  aircraft  are  available  for  mission  taskings. 
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Figure  4.  ISFIM/CBM  Data  Flow 


Viewed  in  another  manner,  the  flow  of  information  in  Figure  4  is  decomposed  to 
show  the  sequence  of  event  flows  within  the  model.  This  flow,  shown  in  Figure  5,  depicts 
where  information  is  transferred  between  model  nodes  and  the  result  of  data  queries 
between  nodes.  Iterative  items  are  shown  in  the  loop  boxes  and  continue  during  missions 
to  ensure  updates  to  system  health  and  processing  are  populated  throughout  the  system. 
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sd  ISHM/CBM  Sequence  J 


Figure  5.  ISFIM/CBM  Sequence  Diagram 


Model  Verification  and  Validation 

Verification  is  the  process  of  determining  if  the  model  is  an  accurate 
representation  of  the  architecture  and  its  processes  and  that  those  processes  contain  the 
correct  information.  The  Arena®  model  in  this  work  was  checked  against  the  system 
architectures  in  Figure  3  and  Figure  6  to  ensure  proper  inclusion  of  processes  and 
decision  points.  Additionally,  the  data  flow  in  Figure  4  and  sequence  in  Figure  5  dictated 
model  information  sharing. 

During  model  development,  checks  for  reasonableness,  i.e.,  inputs  map  to  logical 
outputs,  were  conducted  for  individual  elements  and  processes.  Data  were  also  output  to 
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a  spreadsheet  at  discrete  points  in  the  model  to  ensure  logical  behavior  of  the  modeled 
parameters.  In  the  process  of  conducting  these  checks,  small  errors  in  logic  and  formulas 
were  discovered  and  corrected.  Arena®  has  the  capability  to  animate  models,  which 
allowed  the  author  to  follow  entities  throughout  the  model  processes  to  ensure  proper 
paths  were  followed.  Further,  random  distributions  were  held  as  fixed  values  to  simplify 
checking  of  timing  and  equations  throughout  the  model.  Verification  of  the  finished 
model  was  conducted  through  checks  of  outputs  with  progressively  changing  inputs  to 
verify  logical  shifts  in  outputs. 

Validation  is  process  of  assurance  that  the  data  processed  by  the  model  is 
consistent  with  expected  and  real-world  infonnation  and  is  performed  by  analyzing 
model  simulations.  In  this  work,  validation  is  a  difficult  task  as  the  systems  are 
theoretical  and  randomly  generated  as  are  many  of  the  processes  undergone  in  the  model. 
As  such,  there  are  no  real-world  results  to  compare  the  simulation  data  with  for  validation 
purposes.  Therefore,  what  remains  is  to  validate  model  assumptions  and  random 
distributions  for  processes  to  ensure  their  reasonableness.  Distributions  for  model 
processes  are  meant  to  represent  simple  inspection  and  replacement  tasks  that  can  be 
conducted  in  a  timely  manner  as  opposed  to  an  overhaul  of  an  entire  aircraft,  which 
would  require  considerably  more  time.  The  distributions,  therefore,  are  deemed  valid  for 
this  requirement.  Further,  the  underlying  assumptions  appear  valid  and  discrepancies  are 
annotated  and  discussed. 
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Summary 

The  architecture  and  evaluation  techniques  laid  out  in  this  chapter  establish  the 
processes  and  methods  used  in  the  remainder  of  this  research.  The  establishment  of  a 
system  architecture  early  in  the  design  process  aids  in  ensuring  relevant  processes  and 
procedures  are  captured  and  define  inputs  and  outputs  required  for  successful 
implementation.  The  remaining  chapters  of  this  dissertation  detail  the  research  methods, 
results  and  conclusions  relevant  to  the  implementation  of  a  condition  based  maintenance 
program  enabled  through  integrated  systems  health  management. 
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III.  Prognostic  Uncertainty 


Chapter  Overview 

Across  many  industries,  systems  are  exceeding  their  intended  design  lives, 
whether  they  are  ships,  bridges  or  military  aircraft.  As  a  result  failure  rates  can  increase 
and  unanticipated  wear  or  failure  conditions  can  arise.  Health  monitoring  research  and 
application  has  the  potential  to  more  safely  lengthen  the  service  life  of  a  range  of  systems 
through  utilization  of  sensor  data  and  knowledge  of  failure  mechanisms  to  predict 
component  life  remaining.  A  further  benefit  of  health  monitoring  when  combined  across 
an  entire  platform  is  system  health  management.  System  health  management  is  an  enabler 
of  condition  based  maintenance,  which  allows  repair  or  replacement  based  on  material 
condition,  not  a  set  time.  Replacement  of  components  based  on  condition  can  enable  cost 
savings  through  fewer  parts  being  used  and  the  associated  maintenance  costs.  The  goal  of 
this  research  is  to  show  the  management  of  system  health  can  provide  savings  in 
maintenance  and  logistics  cost  while  increasing  vehicle  availability  through  the  approach 
of  condition  based  maintenance. 

This  work  examines  the  impact  of  prediction  accuracy  uncertainty  in  remaining 
useful  life  prognostics  for  a  squadron  of  12  aircraft.  The  uncertainty  in  this  research  is 
introduced  in  the  system  through  an  uncertainty  factor  applied  to  the  useful  life 
prediction.  An  Arena8  discrete  event  simulation  is  utilized  to  explore  the  effect  of 
prediction  error  on  availability,  reliability,  and  maintenance  and  logistics  processes. 
Aircraft  are  processed  through  preflight,  flight,  and  post-flight  operations,  as  well  as 
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maintenance  and  logistics  activities.  A  baseline  case  with  traditional  time-driven 
maintenance  is  performed  for  comparison  to  the  condition  based  maintenance  approach 
of  this  research. 

This  research  does  not  consider  cost  or  decision  making  processes,  instead 
focusing  on  utilization  parameters  of  both  aircraft  and  manpower.  The  occurrence  and 
impact  of  false  alarms  on  system  performance  is  examined.  The  results  show  the  potential 
availability,  reliability,  and  maintenance  benefits  of  a  health  monitoring  system  and 
explore  the  diagnostic  uncertainty. 

Background 

Across  military  and  commercial  fleets,  aircraft  are  an  example  where  lengthening 
service  lives  and  budget  constraints  can  adversely  affect  safety.  As  a  result,  more 
frequent  inspections  are  required  as  service  life  increases  to  ensure  safety  of  the  users  and 
the  environment.  However,  the  cost  of  large  scale  modifications  or  replacement  in  the 
case  of  hundreds  of  aircraft  is  a  significant  hurdle  to  overcome  in  most  instances  (Shoup 
et  ah,  2011).  The  impact  of  shrinking  budgets  can  also  reduce  inspection  frequency  or 
delay  needed  repairs  in  favor  of  only  performing  mission  critical  tasks  (Roach,  2009). 
Maintenance  strategies  must  change  to  meet  the  extended  in-service  requirements  and  the 
constraints  imposed  by  shrinking  government  and  industry  budgets. 

Condition  based  maintenance  (CBM)  is  an  evolving  maintenance  concept  with  a 
goal  of  reducing  maintenance  and  thus  life  cycle  costs  while  increasing  operational 
availability  made  possible,  in  part,  by  leveraging  health  monitoring  techniques. 
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Department  of  Defense  Instruction  (DoDI)  4151.22  defines  CBM  as  “the  application  and 
integration  of  appropriate  processes,  technologies,  and  knowledge-based  capabilities  to 
achieve  the  target  availability,  reliability,  and  operation  and  support  costs  of  DoD 
systems  and  components  across  their  life  cycle,”  (Under  Secretary  of  Defense  (AT&L), 
May  2008,  p.  1-1).  Integrated  system  health  management  and  its  impact  on  performance, 
cost,  supply  chain  as  well  as  traditional  maintenance  inspections  and  practices  are  the 
focus  of  this  research.  With  the  F-35  maintenance  and  logistics  alone  projected  to  cost 
$1.1  trillion  over  the  55  year  life  span  amid  shrinking  defense  budgets,  the  need  to  reduce 
the  life  cycle  cost  (LCC)  of  military  aircraft  is  paramount  (Shalal-Esa,  2013). 
Additionally,  legacy  aircraft  may  not  be  fitted  with  the  proper  sensors  to  fully  implement 
health  assessment  leading  to  costly  inspections,  in  both  time  and  maintenance  dollars. 
This  reduces  operational  availability  (Ao)  and  the  funds  available  for  other  needs.  CBM 
is  a  demand  driven  maintenance  process  based  on  indications  of  stresses  or  impending 
failure  of  a  component  or  system.  When  appropriately  applied,  CBM  has  the  potential  to 
reduce  life  cycle  cost  and  increase  mission  reliability  by  eliminating  unnecessary 
maintenance  actions  (Butcher,  2000).  Ellis  (2008)  argues  that  cost-effective  systems 
monitoring  allows  repair  actions  based  on  system  condition  rather  than  costly  time -based 
maintenance.  Additionally,  maintenance  may  be  forecast  for  completion  that  minimizes 
impact  on  the  operational  mission  of  the  system.  Secondary  failures,  where  one 
component’s  failure  causes  adverse  perfonnance  or  accelerated  degradation  of 
interrelated  components,  may  also  be  reduced  by  implementing  CBM  as  a  result  of 
prompt  repair  or  replacement  of  the  primary  cause  of  fault. 
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CBM  compares  data  collected  from  vehicle  systems  and  their  components  and 
compares  that  infonnation  with  a  predetermined  threshold  prior  to  failure,  or  to  failure  for 
some  non-critical  components,  then  dictates  repairs  or  replacement  of  parts.  Additionally, 
interim  time-based  inspections  required  under  the  baseline  preventive  maintenance  (PM) 
approach  are  forgone,  or  significantly  reduced  in  frequency,  in  lieu  of  continuous  analysis 
of  the  aircraft  via  the  integrated  systems  health  management  (ISHM)  system. CBM 
requires  sensor  or  inspection  data  to  accurately  diagnose  the  condition  of  a  component. 
Manual  inspections  can  prove  costly  in  terms  of  time  to  perform  if  the  part  requires 
disassembly  or  removal  of  other  components  to  observe  its  condition.  Technology  exists 
for  some,  and  is  under  development  for  other  components,  to  detennine  wear  or 
impending  failure  conditions  in  lieu  of  manual  inspections  (Glaser  et  al.,  2007; 
Speckmann,  2007).  The  data  from  these  health  monitoring  sensors  may  then  be  compiled 
to  predict  remaining  useful  life.  Certainty  is  not  100%,  be  it  in  the  interpretation  of  data 
collected  on  component  condition  or  in  prediction  of  remaining  life  based  on  that  sensor 
data.  This  uncertainty  has  the  potential  to  lead  to  poor  estimation  of  component 
condition,  which  can  result  in  false  conclusions  about  safety  of  flight  decisions  and 
ultimately  to  critical  failures. 

Integrated  Systems  Health  Management  Enabler 

The  benefits  of  ISHM  are  the  abilities  to  reduce  inspection  length,  defer 

maintenance  and  migrate  to  maintenance  on  demand  with  the  end  goal  to  increase 

operational  availability  through  reduced  maintenance  time  (Speckmann,  2007).  Applying 

ISHM  enables  CBM  as  opposed  to  preprogrammed  periodic  maintenance  practices;  that 
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is,  maintaining  only  when  required  instead  of  when  prescribed  by  schedules,  thus 
optimizing  maintenance  labor  (Roach,  2009).  SHM  technologies  and  resulting  modified 
maintenance  programs  serve  to  reduce  the  total  life-cycle  cost  of  a  system  and  increase 
availability.  While  this  may  drive  increased  acquisition  cost  of  a  weapon  system  or 
aircraft  due  to  the  inclusion  of  health  monitoring  systems,  the  goal  is  to  offset  the 
increase  with  reduced  operations  and  maintenance  costs  over  the  life  of  the  program. 
Published  literature  shows  the  savings  potential  of  ISHM  enabled  condition  based 
maintenance  on  aircraft  life  cycle  cost: 

•  40%  for  vehicle  maintenance  (Walls  et  ah,  1999) 

•  30%  to  50%  for  fuselage  panels  (Pattabhiraman  et  ah,  2010) 

•  10%  electrical  components  (Scanff  et  ah,  2007) 

•  50-80%  for  the  Boeing  777  (Gorinevsky,  Gordon,  Beard,  Kumar,  &  Chang, 
2005). 

In  general,  an  application  project  could  choose  to  increase  the  detection 
capability,  accepting  a  higher  acquisition  cost  with  the  goal  of  lowering  the  overall 
system  life  cycle  cost  through  more  efficient  operations  and  maintenance.  For  a  given 
detection  system,  however,  increasing  the  detection  capability  (e.g.,  lowering  a  threshold) 
will  come  at  the  expense  of  a  degraded  false  alann  rate;  the  two  are  competing  objectives. 
Ultimately,  the  value  of  the  prognostic  system  will  depend  on  the  achievable  balance 
between  detectability  for  safety  concerns  and  acceptable  false  alarm  rates  to  avoid 
unnecessary  and  expensive  maintenance  actions.  Aircraft,  or  other  vehicle,  availability  is 
linked  to  the  balance  of  sensor  reliability  and  detectability  and  the  capability  of  the 
system  to  decrease  maintenance  duration  (Hoyle  et  ah,  2007). 
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It  is  important  to  understand  that  uncertainty  will  exist  in  the  diagnosis  and 
prognosis  of  system  health.  Numerous  points  of  entry  exist  for  uncertainty  to  work  its 
way  into  remaining  useful  life  (RUL)  prediction.  Component  performance  data  is 
dependent  on  sensor  health  and  accuracy.  It  is  also  difficult  to  anticipate  the  exact 
conditions,  load,  environment,  etc,  that  the  vehicle  or  machine  will  undergo  during 
operation  or  storage.  Quantifying  and  compiling  these  uncertainties  is  a  difficult  task 
individually  and  made  harder  by  potential  amplifying  effects  on  each  other. 
Sankararaman  and  Goebel  (2013)  discuss  factors  of  uncertainty  in  RUL  prediction  and 
lay  out  methods  to  quantity  and  interpret  the  sources.  They  also  stress  the  need  to 
accurately  determine  the  uncertainty  in  the  prediction  for  the  prediction  to  be  of  use.  The 
goal  is  that  the  probabilistic  estimates  of  RUL  based  on  real  time  monitoring  allow 
increased  time  to  accumulate  on  parts,  thus  increasing  the  MTBF  for  the  ISHM  aircraft 
and  generating  savings  through  fewer  spares  procurements  or  repair  actions. 

Determining  the  effectiveness  of  system  health  monitoring  approaches  requires  a 
method  for  comparison  of  techniques.  The  remainder  of  this  paper  discusses  modeling 
approaches,  evaluation  techniques  and  results  of  this  research. 

Modeling  Approaches 

Research  into  the  effects  of  prognostics  on  integrated  logistics,  maintenance  and 
aircraft  systems  frequently  neglects  the  impact  of  uncertainty  on  HM  model  outcomes. 
Rebulanan  utilizes  a  discrete  event  simulation  to  represent  the  F-35  autonomic  logistic 
system  (ALS)  system  with  a  health  management  system,  LRUs,  communication  system, 
supply,  and  maintenance  systems  (Rebulanan,  2000).  Rebulanan  further  evaluates 
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performance  with  aircraft  availability,  mission  capable  and  non-mission  capable  rates, 
and  mission  reliability.  Rebulanan’s  model  shows  sensitivity  of  the  supply  wait  time  to 
the  detection  lead  time  for  an  impending  failure  and  the  supply  stock  levels.  This 
outcome  is  somewhat  intuitive  in  that  as  the  prognosis  of  an  impending  failure  is  detected 
earlier  and  with  greater  accuracy,  the  supply  system  can  plan  further  in  advance,  ensuring 
parts  are  available  when  required. 

Rodrigues  and  Yoneyama  (2012;  2013)  explore  the  effect  of  prognostics  on  spare 
parts  inventories  for  both  repairable  and  non-repairable  systems  compared  with 
conventional  supply  processes.  Both  studies,  simulated  over  15  years  each,  show  cost 
savings  for  the  ISHM  enabled  system  over  the  conventional  one.  In  their  work  on  non- 
repairable  items  they  discuss  uncertainty  in  failures  and  their  impact  on  supply  policy,  but 
they  do  not  include  the  impact  of  prognostic  uncertainty  on  maintenance  operations  for 
false  alann  adjudication  or  aircraft  operational  availability.  Similarly,  while  they  do 
address  prognostic  error  in  repairable  systems  they  focus  on  the  impact  of  sparing  to 
account  for  fleet  availability  without  addressing  false  alarms  and  how  they  might  drive 
costs.  Both  works  provide  an  excellent  analysis  of  the  cost  impact  of  sparing  decisions 
based  upon  health  monitoring  information.  Out  of  stock  costs  are  difficult  to  quantify  but 
do  impact  downtime  for  supply,  which  is  where  the  impact  is  captured  in  our  research 
model.  A  limitation  of  the  nor-repairable  study  is  that  only  one  item  is  investigated, 
leaving  interactions  of  multiple  components  in  question. 

Kahlert,  Giljohanan,  and  Klingauf  (2014)  utilize  a  MATLAB  discrete  event 
simulation  to  analyze  one  Lufthansa  A320  component  with  100%  unscheduled 
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replacement.  They  utilize  process  times,  reliability,  prognostic  accuracy,  and  cost  to 
evaluate  PHM  system  performance.  Additionally,  the  use  of  historic  Lufthansa 
maintenance  data  provides  added  realism  in  the  research.  The  research  focus  only  extends 
for  two  weeks  around  a  replacement,  thus  leaving  out  some  potential  for  a  false  alarm 
condition  to  exist  prematurely.  One  of  their  final  conclusions  is  a  realistic  PHM  system 
could  save  approximately  20%  of  annual  fleet  operation  costs. 

Model  description 

In  this  research,  an  Arena1'  discrete  event  simulation  is  utilized  to  represent  a 
squadron  of  12  aircraft  and  their  associated  mission,  maintenance  and  supply  processes 
over  a  15  year  duration.  This  model  explores  the  impacts  to  this  squadron  in  analyzing  a 
model  containing  elements  not  addressed  in  the  works  of  section  2.  The  authors  add 
uncertainty  not  found  in  Rebulanan’s  work  with  an  interaction  of  multiple  components 
missing  from  Rodriques  and  Yoneyama. 

Model  Components  and  Architecture 

The  initial  component  failure  properties  were  randomly  generated  from  a 
uniform(250,1000)  distribution  for  parts  A-T.  These  times  are  then  utilized  for 
component  replacements  in  the  model.  Each  aircraft  is  generated  and  assigned  20 
components  with  a  failure  time  randomly  sampled  from  an  exponential  distribution,  with 
mean  time  between  failure  (MTBF)  given  in  Table  1,  and  with  probability  density 

—  X/ 

function:  f(x)  =  y^e  /p ,  for  x  >  0  .  The  exponential  distribution  is  chosen  as  a 
representative  reliability  function  for  the  components  for  simplicity  in  model  calculations 
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of  the  constant  failure  rate.  The  model  can  readily  accept  another  failure  distribution 
with  other  components. 


Table  4.  Components  Failure  Times 


Part 

MTBF  (hours) 

A 

502 

B 

280 

C 

775 

D 

750 

E 

763 

F 

364 

G 

441 

H 

829 

I 

769 

J 

941 

K 

778 

L 

363 

M 

272 

N 

642 

0 

696 

P 

268 

0 

822 

R 

585 

S 

996 

T 

842 

The  sampled  failure  times  are  considered  “truth”  in  terms  of  component  failure 
times.  That  is,  if  the  line  replaceable  unit  (LRU)  incurs  more  than  the  associated  failure 
time  in  hours  without  being  repaired  or  preemptively  replaced  as  a  result  of  scheduled 
preventive  maintenance,  overhaul  in  the  baseline  case  or  ISHM  indicated  replacement  in 
the  prognostic  case,  a  failure  occurs.  Aircraft  flow  through  preflight  processing  and 
mission  preparation  prior  to  actually  flying  an  assigned  mission.  The  ISHM  system 
performs  a  scan  to  determine  if  the  aircraft  is  anticipated  to  have  enough  useful  life  to 
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complete  the  mission.  Each  component  decreases  its  life  only  during  engine  running 
operations:  taxi,  take-off,  flying,  landing  and  parking.  In  this  work,  it  is  assumed  that 
LRUs  operate  until  failure.  These  processes  are  visually  depicted  in  Figure  6. 

After  sortie  completion,  diagnostics  are  again  performed  and  in  the  baseline  case, 
maintenance  is  performed  as  well.  ISHM  aircraft  perform  post  flight  scan  and  if 
acceptable  are  released  for  next  flight.  Baseline  aircraft  are  inspected  and  checked  for 
LRU  preventive  maintenance  time.  If  PM  is  not  required,  routine  maintenance  and 
inspections  are  performed  and  the  aircraft  released  for  next  mission.  Aircraft  are  then 
either  parked  until  their  next  mission  or  turned  for  another  flight. 

In  the  maintenance  module,  the  number  of  indicated  failures  is  recorded  and  the 
maintenance  clock  starts.  A  detailed  inspection  is  perfonned  for  both  the  ISHM  and 
baseline  cases,  though  shorter  for  the  ISHM  case.  False  alarms  are  recorded  and  in  the 
ISHM  case  if  a  false  alarm  threshold  over  the  lifetime  of  the  part  is  reached,  the  ISHM 
system  undergoes  maintenance.  The  model  indicates  a  false  alarm  condition  if  the 
predicted  component  RUF  is  less  than  the  “truth”  remaining  time  minus  a  safety  factor 
and  the  anticipated  sortie  duration.  In  the  baseline  case  supply  stock  is  reduced  and  if  not 
in  stock  the  aircraft  is  grounded  until  the  part  arrives.  Parts  are  processed  by  supply 
(occurs  simultaneously  with  other  aircraft  operations  in  the  ISHM  case)  and  transferred 
from  supply  to  maintenance.  Aircraft  are  maintained  and  LRU(s)  life  characteristics  are 
resampled  from  the  failure  distribution(s)  in  Table  1.  The  aircraft  repair  is  checked  and 
the  vehicle  is  routed  back  into  the  mission  queue.  In  the  ISHM  case,  if  the  standby  time 
until  the  next  mission  is  greater  than  the  mean  time  to  perfonn  any  outstanding 
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maintenance  actions,  the  aircraft  is  routed  to  be  maintained  so  as  not  to  impact  mission 
operations.  In  the  baseline  case,  unless  the  part  is  scheduled  for  preventive  maintenance 
the  condition  is  not  known  thus  the  need  for  repair  or  replacement  is  unanticipated  and 
the  aircraft  continues  normal  mission  operations.  Maintenance  actions  are  perfonned 
serially  on  each  aircraft,  that  is,  only  one  inspection  or  maintenance  action  at  a  time, 
continuing  until  all  required  actions  are  complete.  This  assumption  likely  over  constrains 
maintenance  personnel  actions,  leading  to  slightly  higher  maintenance  delays,  but  is  done 
for  model  simplicity  and  has  the  same  effect  on  the  baseline  and  health  monitoring  cases. 
It  is  assumed  that  all  component  inspection  times  for  indicated  or  actual  failures  are 
triangularly  distributed  (20,  30,  45)  minutes  and  LRU  replacements  triangularly 
distributed  (60,  90,  240)  minutes.  These  times  were  chosen  to  represent  a  range  of  repairs 
and  inspections  while  not  portraying  items  which  may  require  multiple  days  to  maintain. 
Additionally,  in  this  research  required  personnel  for  maintenance  actions  are  always 
considered  available.  LRUs  are  always  replaced  when  they  are  serviced. 

Supplies  are  input  into  the  model  at  an  initial  stock  level  and  a  reorder  point.  In 
both  the  baseline  and  ISHM  cases,  the  stock  level  and  reorder  points  are  fixed  for  the 
simulation.  The  levels  are  discussed  further  in  section  3.2.  Once  reorder  point  is  reached, 
the  difference  between  stock  level  and  reorder  point  is  ordered.  Time  between  order  and 
delivery  is  log-normally  distributed  (2,1)  days  for  all  parts.  Additionally,  a  processing 
time  upon  receipt  is  incurred. 
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Figure  6.  ISHM  System  Architecture 
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If  RUL  is  within  a  10  hour  safety  factor  from  failure  the  aircraft  is  routed  to 
maintenance.  If  RUL  is  within  a  prescribed  lead  time  window,  a  supply  check  is 
performed  and  if  parts  aren’t  in  stock  they  are  ordered  to  meet  predicted  maintenance 
activities.  If  RUL  is  within  a  defined  maintenance  window,  component  service  can  occur 
if  parts  are  in  stock  or  the  aircraft  can  continue  flying  missions  if  there  is  sufficient  RUL. 

Sensor  and  Prognostics  Process 

The  ISHM  routine  begins  by  computing  the  remaining  useful  life  (RUL)  of  each 
component.  The  RUL  prognosis  has  two  components,  the  diagnosis  from  the  HM  system 
and  the  prediction  uncertainty.  In  this  research  component  diagnostics  is  taken  as  perfect, 
i.e.,  sensor  always  knows  exact  health.  In  new  components  sensor  diagnostics  can  have 
difficulty  detecting  the  health  state,  thus  providing  data  that  may  not  be  useful.  As  failure 
becomes  more  imminent,  sensor  diagnostics  can  provide  a  more  exact  condition 
diagnosis.  The  resulting  determination  leads  to  component  RUL  being  predicted  as: 

LRU  RUL-  \ogtiovma\(Diagnosis,  uncertainty)  ( 1 ) 

Where  Diagnosis  is  the  log  mean  and  equivalent  to  the  true  remaining  life  and, 
uncertainty  is  the  log  standard  deviation  defined  in  Eq.  (2). 

Uncertainty  is  varied  in  this  research  to  determine  the  impact  of  uncertain 
prognostics  on  Ao  and  sortie  rates.  Uncertainty  is  calculated  as: 

uncertainty  =  In  (Part  RUL)*  uncertainty  factor  (2) 

Where  Part  RUL  is  the  previous  RUL  prediction  for  that  part  and,  uncertainty  factor  is  a 
design  variable. 
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This  information  is  sent  to  the  CBM  module  where  maintenance  predictions  are 
performed.  While  no  specific  RUL  prognostic  technique  is  used,  the  technique  above  is 
utilized  to  represent  compounded  error  or  uncertainty  built  up  in  the  system.  Initially, 
RUL  estimation  is  chiefly  impacted  by  the  uncertainty  factor,  but  in  section  4.2, 
additional  degradation  to  the  system  is  added  to  account  for  sensor  diagnostic  losses.  Eqs. 
(1)  and  (2)  are  representative  equations  developed  by  the  authors  to  portray  the  behavior 
of  health  monitoring  systems.  They  are  not  intended  to  mimic  the  perfonnance  of  a 
particular  system,  but  to  represent  the  functionality  of  a  monitoring  system.  The 
uncertainty  factor  is  a  representation  of  the  accumulated  variability  in  the  prognostics  for 
remaining  useful  life.  This  work  ranges  the  uncertainty  factor  from  a  low  of  0,  to 
represent  perfect  prognosis,  to  a  high  of  100,  which  approaches  half  the  MTBF  of  some 
parts.  Examining  a  range  of  variability  between  these  end  points  allows  system  designers 
to  quantify  how  much  uncertainty  is  acceptable  in  a  health  monitoring  system  before 
selecting  one  for  inclusion  on  an  aircraft. 

The  system  then  enters  a  decision  node  where  the  RUL  is  compared  to  a  set  safety 
factor,  which  would  be  a  policy  decision  based  on  mission  requirements.  If  there  is  RUL 
above  the  safety  factor  and  the  projected  sortie  length  does  not  encroach  on  the  safety 
factor,  the  aircraft  is  cleared  for  flight.  If  the  RUL  is  below  the  safety  factor,  the 
component(s)  are  flagged  and  sent  to  maintenance.  If  RUL  is  sufficient,  the  aircraft  is 
cleared  for  the  next  process.  In  all,  the  aircraft  is  checked  prior  to  mission  preparation 
(fuelling  and  cargo  loading),  prior  to  take-off,  during  flight,  and  upon  landing.  If  all  of 
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these  checks  are  satisfactory  the  aircraft  continues  through  missions  and  standby  time 
until  a  maintenance  action  is  required. 

The  CBM  system  preorders  parts  to  meet  demands  as  described  above.  If  the  part 
is  not  in  stock,  the  aircraft  is  placed  in  a  non-mission  capable  supply  hold  until  the  part 
arrives.  Upon  maintenance  completion,  the  ISHM  equipped  aircraft  bypasses  additional 
check-outs  normally  performed  to  inspect  work,  instead  relying  on  the  ISHM  system  to 
perform  them.  The  aircraft  is  then  released  for  the  next  mission  tasking. 

Evaluation  Parameters 

Establishment  of  useful  perfonnance  measures  to  evaluate  the  model  is  essential. 
To  that  end,  metrics  currently  used  to  determine  aircraft  and  system  perfonnance  are 
preferred  as  a  means  of  comparison.  Three  categories  of  metrics,  although  interwoven, 
are  laid  out  below  and  are  used  when  discussing  the  results  of  this  research:  availability; 
reliability;  and  maintenance. 

Availability 

To  understand  operational  availability  and  why  it  is  a  good  measure  of  system 
performance  for  this  model,  it  is  useful  to  be  familiar  with  achieved  and  inherent 
availabilities  as  well. 

Inherent  availability  (A;)  is  the  availability  of  a  system  operating  under  an  ideal 
support  system.  This  means  delays  for  logistics,  administrative  delays  and  preventive 
maintenance  time  are  excluded,  leaving  only  operating  time  and  corrective  maintenance. 

Achieved  Availability  (Aa)  adds  preventive  maintenance  to  A;  in  addition  to 

corrective  maintenance.  Logistics,  supply  and  administrative  delays  are  ignored  and  those 
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assets  are  assumed  to  be  instantaneously  available  when  required.  Achieved  availability 
is  detennined  examining  the  mean  time  between  maintenance,  MTBM,  and  the  mean 
maintenance  time  (MMT). 

Operational  availability  (A0)  adds  the  final  piece  to  the  downtime  portion  of  the 
equation.  A0  includes  logistics,  supply  and  administrative  delays  to  the  PM  and  CM  for 
the  system  resulting  in  the  mean  downtime  for  the  system.  Operational  availability  is  the 
system  availability  the  user  of  a  system  realizes,  (ReliaSoft,  2007).  Mathematically, 
operational  availability  is: 

_  Uptime  _  MTBM 

Uptime  +  Downtime  MTBM  +  MMT  +  MLDT 

Where  MLDT  is  the  mean  logistics  delay  time. 

Eq.  (3)  is  not  the  only  way  to  define  operational  availability.  Pryor  (2008) 
discusses  methods  to  calculate  A0  seen  in  Eq.  (4)  using  the  uptime/(uptime  +  downtime) 
definition  of  Eq.  (3),  but  the  definition  is  slightly  different. 


_ OT  +  ST _ 

OT  +  ST  +  TPM  +  TCM  +  TALDT 


(4) 


Where  OT  is  the  operational  time,  ST  is  the  standby  time,  TPM  is  the  total  preventive 
maintenance  time,  TCM  is  the  total  corrective  maintenance  time,  and  TALDT  is  the  total 
administrative  and  logistics  delay  time,  equivalent  to  MLDT. 

Figure  7  shows  the  components  of  up  and  downtimes.  This  is  by  no  means  an 
exhaustive  list  and  further  breakdowns  are  possible,  especially  in  the  administrative  and 
logistics  delay  blocks,  but  for  this  research  these  components  define  the  temporal 
parameters. 
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Time 


Figure  7.  Components  of  System  Usage  Time  (Pryor,  2008) 


A  function  of  a  system’s  operational  availability,  average  daily  flying  hours  is  a 
measurement  of  the  ability  of  the  squadron  as  a  whole  to  perform  the  assigned  missions. 
Further,  the  number  of  sorties  flown  per  day  is  a  function  of  the  mission  requirements, 
but  also  the  performance  of  the  aircraft  as  well  as  maintenance  and  logistics  systems. 

Reliability 

In  the  commercial  environment,  up  and  downtimes  can  also  be  assigned  costs  as 
the  systems  impact  revenue  generation.  Kahlert,  Giljohanan  and  Klingauf  discuss 
dispatch  reliability,  or  the  “ratio  of  revenue  departures  without  delay  or  cancellations 
compared  to  all  flights,”  (2014,  p.l).  They  go  on  to  summarize  commercial  aircraft  cost 
accounting  for  delays  and  cancellations.  Downtime  has  an  associated  cost  beyond 
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maintenance  labor  in  lost  revenue.  Similarly,  uptime  has  the  potential  to  generate 
revenue,  when  not  in  a  standby  capacity.  For  military  systems,  assigning  costs  to  up  and 
downtime  is  problematic  as  there  is  no  profit  to  generate  and  supporting  national  security 
is  difficult  to  assign  a  value  to.  In  essence,  military  aircraft  are  consumptive,  always 
operating  at  a  loss.  Policy  and  research  can,  however,  strive  to  reduce  these  consumption 
costs. 

False  alarms  diagnosed  or  predicted  by  the  ISHM  system  drive  unnecessary 
maintenance  and  supply  actions  as  well  as  placing  an  otherwise  mission  capable  aircraft 
into  a  NMC  state.  These  maintenance  and  supply  actions  increase  the  overall  cost  impact 
of  the  ISHM  system  as  they  are  not  free.  A  key  requirement  for  successful  deployment  of 
an  ISHM  architecture  enabling  CBM  is  a  low  false  alann  rate  with  reliable  detection 
(Ellis,  2008;  A.  Van  Horenbeek  et  ah,  2013).  False  alarms  in  the  baseline  model  result 
from  CND  and  RTOK  discussed  previously.  Totals  for  each  of  the  models  will  be 
recorded  for  comparison.  Additionally,  an  increase  in  false  alarms,  above  a 
predetennined  threshold,  on  an  aircraft  with  an  ISHM  system  will  trigger  an  inspection  of 
the  ISHM  system  sensors  providing  erroneous  data  and  potentially  of  the  ISHM  system 
logic  itself. 

The  ability  to  tolerate  false  alarms  is  a  two-fold  evaluation.  First,  the  cost 
associated  with  each  false  alann  shrinks  any  cost  benefit  of  the  ISHM  system  over  the 
baseline  system.  Second,  too  many  false  alanns  can  trigger  a  “cry  wolf’  attitude  towards 
the  system  or  result  in  wasted  time  maintaining,  or  checking  the  system  thus  decreasing 
the  operational  availability  of  the  aircraft  and  the  reliability  of  the  ISHM  system.  For  an 
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ISHM  architecture  to  be  effective  it  cannot  trigger  excessive  false  alarms  which,  in  turn, 
trigger  maintenance  actions  on  the  system. 

Maintenance  and  Logistics 

Inspection  intervals  are  time-driven  processes  under  the  baseline  aircraft  case  and 
are  prescribed  to  monitor  systems  for  indications  of  damage.  They  are  generally  based  on 
historic  or  predicted  failure  data  and  are  conducted  to  ensure  early  indications  of  failure 
are  discovered  before  they  catastrophically  fail  the  system  or  adjacent  components.  An 
assumption  for  this  research  is  that  all  systems  of  interest  on  the  aircraft  are  monitored  in 
the  ISHM  model.  If  that  were  not  the  case  time -based,  but  informed  through  ISHM 
inferences,  inspections  would  still  be  required.  In  this  research,  the  ISHM  case  only 
requires  inspection  upon  indication  of  failure  or  impending  failure  by  the  system. 
Therefore,  the  inspection  intervals  should  be  further  apart  and  of  shorter  duration  for 
ISHM  than  for  time-based  methods.  The  preprogrammed  PM  inspections  of  the  baseline 
are  defined  based  on  operating  hours. 

Accounting  for  the  required  time  to  repair  and  inspect  aircraft  is  critical  in 
determining  the  impacts  of  system  changes  to  downtime  and  manpower  costs.  In  addition 
to  the  repair  of  malfunctioning  components,  inspections  based  upon  fault  indications, 
either  in  perfonnance  or  indicated  by  the  ISHM  system,  drive  mission  unavailability  and 
decrease  system  performance  metrics.  A  common  metric  is  to  measure  the  required 
maintenance  man  hours  per  aircraft  flight  hour  or  MMH/FH.  This  factor  can  then  be 
utilized  in  forecasting  manpower  requirements  and  required  downtime-based  on  mission 
requirements.  Similarly,  mean  downtime  (MDT),  the  average  amount  of  time  it  takes  to 
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return  an  aircraft  to  flying  status  once  a  fault  is  indicated,  is  a  commonly  used 
maintenance  performance  metric. 

Supply  delay  is  the  time  between  actual  part  need  and  when  the  supply  system 
delivers  the  part  to  maintenance  and  will  impact  both  the  baseline  and  ISHM/CBM  cases. 
Non-mission  capable  supply  (NMCS)  is  the  common  measure  of  this  supply  delay.  The 
prognostic  CBM  case  will  anticipate  failure  and  sparing  requirements  further  out  from 
maintenance  demand  and  allow  for  advanced  ordering  if  stock  levels  are  inadequate.  The 
current  baseline  process  relies  on  anticipating  failures  and  providing  stock  levels  at 
individual  bases  or  in  some  cases  a  central  location  that  can  be  tasked  to  deliver  spares 
when  required.  This  process  increases  the  logistic  footprint  by  requiring  storage  facilities 
for  materiel  that  may  not  be  needed  for  upwards  of  a  year.  Managing  these  spares  and  the 
facility  requires  additional  resources,  manpower  and  money.  “Logistics  response  time,  a 
measure  of  supportability  and  an  indirect  measure  of  readiness,”  (Deputy  Under 
Secretary  of  Defense  for  Logistics  and  Materiel  Readiness,  May,  2008,  p.  6-4),  drives 
shorter  maintenance  times  and  as  such  impacts  supply  and  maintenance  downtime. 

While  maintenance  policy  and  cost  decisions  impact  LRU  replacement  decisions, 
the  prognostics  capability  plays  an  important  role  in  determining  when  to  repair  or 
exchange  components.  Confidence  in  the  performance  of  the  diagnostics  and  prognostics 
systems  could  lead  to  a  decreasing  safety  factor  as  to  when  maintenance  occurs.  This 
resulting  increase  in  useable  time  of  each  part  saves  money  through  extended  service  life 
for  the  components  and  reduces  the  amount  of  supplies  consumed.  Capturing  the  amount 
of  useful  life  lost  for  the  components  can  quantify  the  gains  that  may  be  achievable. 
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Model  Variables 


This  research  explores  the  impact  of  RUL  prediction  uncertainty  on  the 
availability,  reliability,  and  maintenance  and  logistics  categories  above.  Evaluation  of  the 
model  is  accomplished  through  simulation  of  15  years  of  aircraft  utilization.  Further,  two 
design  cases  are  initially  utilized  in  the  simulations.  The  remaining  useful  life  uncertainty 
factor  is  varied  at  14  levels  with  two  false  alarm  limits  at  0  and  10000  and  the  model 
assessed  at  each  increment.  The  levels  for  the  FA  limit  is  meant  to  indicate  that  at  0,  the 
ISHM  system  is  always  maintained  after  a  false  alarm  and  at  10000,  policy  allows  nearly 
unlimited  false  alanns  by  the  ISHM  system  before  requiring  repair.  These  levels  are 
found  in  Table  5.  At  each  uncertainty  factor  100  simulations  are  run  to  establish 
confidence  in  the  results,  and  the  means  of  these  data  are  presented.  Sensitivity  to  values 
of  FA  limit  greater  than  0  is  presented  later  in  this  paper  once  sensor  and  prognostics 
degradation  are  considered.  Additionally,  two  simulations  of  the  baseline  case  with  no 
prognostics  are  run  where  component  stock  levels  are  varied. 

Stock  levels  for  the  ISHM  case  are  held  to  1  nominally  and  ordered  as  predicted 
by  the  system.  In  the  baseline  case,  two  comparisons  are  examined,  one  where  the  stock 
levels  are  kept  the  same  as  the  ISHM  case.  The  other  stock  level  case  holds  4  parts  in 
stock  and  reorders  when  the  level  drops  to  2.  This  variance  of  stock  level  for  the  baseline 
case  makes  the  process  comparable  to  minimal  levels  as  in  the  ISHM  case  and  robust 
levels  when  failure  is  somewhat  uncertain. 
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Results 


No  ISHM  Degradation  Results 

Daily  flying  hour  averages  for  all  simulation  runs  are  located  in  Table  5.  It  is 
noted  in  these  data  that  a  decrease  of  19.04  flying  hours  per  day  occurs  over  the  range  of 
uncertainty  factors  for  a  FA  limit  of  0.  This  decrease  is  smaller  when  the  FA  limit  is 
10000,  reaching  3.45  hours.  This  reduction  corresponds  to  6949  and  1261  hours 
respectively  in  lost  flying  each  year,  the  equivalent  of  removing  more  than  1  aircraft’s 
missions  from  the  flight  taskings  in  the  unlimited  case  and  over  5  aircraft  in  the  0  FA 
limit  case.  The  last  two  rows  in  Table  5  contain  performance  results  of  the  baseline  model 
where  the  numbers  in  parentheses  represent  the  stock  level  and  reorder  point  respectively. 
For  the  baseline  model,  the  (1,0)  supply  case  yields  only  18.36  daily  flying  hours  while 
the  (4,2)  case  achieves  27.94  hours.  The  chief  cause  of  this  difference  is  attributed  to  the 
(1,0)  case  waiting  for  supplies  to  be  delivered  as  they  are  only  ordered  as  needed  and 
only  1  item  is  held  in  stock.  The  ISHM  cases  all  benefit  from  the  prognostic  capability  of 
the  ISHM  system  in  ordering  supplies  to  meet  requirements. 

A  typical  measure  when  examining  the  maintenance  demand  of  an  aircraft  is 
maintenance  man  hours  per  flying  hour.  Figure  8  examines  MMH/FH  for  the  case  where 
all  false  alarms  trigger  ISHM  system  maintenance  and  the  case  where  FAs  in  the  system 
do  not  incur  ISHM  maintenance,  merely  downtime  to  adjudicate  the  alann  does  not 
require  maintenance.  As  shown  in  Figure  8,  the  0  FA  limit  case  MMH/FH  increases 
linearly  as  the  uncertainty  factor  increases.  This  growth  results  from  the  number  of 
maintenance  actions  on  the  ISHM  system  as  every  FA  triggers  ISHM  maintenance. 
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Maintaining  the  ISHM  system  takes  more  time  than  merely  adjudicating  a  false  alarm  by 
the  ISHM  system  thus  the  increase  in  maintenance  hours.  In  the  case  where  FAs  do  not 
trigger  ISHM  repair,  the  MMH/FH  grow  slowly  reaching  a  maximum  of  0.268  vs.  4.198 
for  the  0  FA  case.  Inspection  and  maintenance  times  drive  the  maintenance  hours  and  if 
inspection  times  were  to  increase  significantly,  the  number  of  false  alarms  shown  in 
Figure  9  could  change  the  behavior  of  Figure  8.  Additionally,  as  the  uncertainty  factor 
increases  more  false  alarms  occur  as  shown  in  Figure  9  as  does  the  resulting  downtime 
associated  with  the  false  alarms  observed  in  Figure  10.  For  comparison,  the  baseline 
cases  have  MMH/FH  ratios  of  0.546  and  0.549  for  the  (1,0)  and  (4,2)  cases  respectively. 
In  the  baseline  case,  time -based  preventive  maintenance  occurs  at  set  intervals  versus  the 
condition  based  method  employed  by  CBM  driving  extra  maintenance  hours. 

Table  5.  Average  Daily  Flying  Hours 


ISHM  False  Alann  Limit 

0 

10000 

Uncertainty 

Mean  Daily  Flying  Hours 

0 

35.98 

35.98 

2 

35.25 

35.26 

5 

34.90 

35.00 

7 

34.36 

34.92 

10 

33.53 

34.86 

20 

30.14 

34.37 

30 

27.32 

34.10 

40 

24.88 

33.85 

50 

22.96 

33.55 

60 

21.38 

33.25 

70 

19.93 

33.15 

80 

18.92 

32.92 

90 

17.80 

32.62 

100 

16.94 

32.52 

Baseline 

18.36 

Baseline 

27.94 
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While  the  MMH/FH  numbers  are  low  for  an  entire  aircraft,  for  a  system  of 
subcomponents  when  scaled  up  it  is  feasible.  For  example,  the  U.S.  Air  Force  C-17  fleet 
operates  around  6  MMH/FH  (Nelms,  2008). 


Uncertainty  Factor 

Figure  8.  Maintenance  Man  Hours  per  Flight  Hour 

Figure  9  illustrates  the  average  false  alanns  per  aircraft  per  year.  The  quantity 

increases  from  0  for  the  0  uncertainty  factor,  perfect  prognosis,  case  to  101.52  and  196.65 

for  the  0  and  unlimited  FA  cases  respectively  at  the  100  uncertainty  factor  case.  As 

observed  in  the  figure,  since  the  amount  of  time  spent  in  maintenance  repairing  the  ISHM 

system  for  every  FA  in  the  0  limit  case  increases  as  the  uncertainty  factor  increases  the 

number  of  false  alarms  is  lower.  It  should  be  noted  that  this  is  not  a  reduction  in  the  FA 

rate,  as  the  prognosis  accuracy  is  not  degrading  over  time  for  this  initial  investigation. 

This  mostly  results  from  the  maintenance  time  taking  away  time  when  the  aircraft  could 
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be  flying  and,  as  noted  in  Table  5,  the  mean  daily  flying  hours  are  nearly  double  for  the 
unlimited  FA  case. 
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Figure  9.  False  Alarms  per  Aircraft  per  Year 


While  the  number  of  false  alarms  per  aircraft  per  year  is  nearly  doubled  in  the  0 
limit  case  versus  the  no  limit  case,  FA  downtime  increases  at  a  considerably  higher  rate. 
As  shown  in  Figure  10,  the  average  downtime  each  aircraft  experiences  per  year  due  to 
FA  increases  from  0  for  the  perfect  prognosis  case  to  111.31  hours  for  the  unlimited  FA 
case  and  over  2000  hours  for  the  FA  limit  0  case.  The  increase  is  attributed  to  the 
additional  maintenance  required  to  maintain  the  ISHM  system  at  the  lower  FA  trigger. 
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Figure  10.  False  Alarm  Downtime  per  Aircraft  per  Year 


Compiling  all  components  of  downtime  and  the  number  of  times  the  aircraft  is 
down  for  maintenance  leads  to  the  mean  downtime  for  an  occurrence.  As  shown  in 
Figure  11,  mean  downtime  decreases  from  3.24  hours  when  the  uncertainty  prognosis  is 
perfect  to  a  low  1.09  hours  when  the  uncertainty  factor  is  100  and  FA  limit  is  unlimited. 
This  decrease  is  attributed  to  the  fact  that  while  the  aircraft  is  being  removed  from  service 
more  often  to  adjudicate  false  alarms  as  the  uncertainty  factor  increases,  the  inspections 
do  not  take  as  long  as  the  aircraft  is  quickly  returned  to  operation.  MDT  for  the  0  FA 
limit  case  grows  as  the  uncertainty  factor  rises,  mostly  due  to  all  components  requiring 
inspection  and  sensor  repair  for  each  time  down.  As  uncertainty  rises,  the  aircraft  is 
brought  down  more  frequently,  but  more  often  for  a  false  alarm  than  maintenance 
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actions.  Adjudicating  a  false  alann  through  inspection  takes  less  time  than  a  repair,  thus 
the  downtime  is  smaller.  For  the  baseline  (1,0)  case,  MDT  is  171.82  hours,  and  for  the 
(4,2)  case  17.47  hours.  The  MDT  for  the  (1,0)  case  is  high  mainly  due  to  NMCS  as  there 
is  only  a  stock  level  of  1  LRU  and  parts  are  ordered  on  demand,  not  schedule.  The  other 
major  driver  for  the  baseline  MDT  is  the  PM  process. 


Uncertainty  Factor 


Figure  11.  Mean  Downtime 


At  the  low  end  of  the  uncertainty  factor  range,  the  fixed  10  hour  safety  factor 
imposed  on  each  part  accounts  for  a  majority  of  the  lost  life  each  LRU,  with  the 
remainder  mostly  coming  from  the  component  not  being  able  to  safely  cover  the 
projected  sortie  duration.  As  the  uncertainty  factor  increases,  the  mean  life  lost  per 


component  increases  as  well  due  to  the  uncertainty  in  the  RUL  prediction  necessitating 
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replacement  before  LRU  failure.  Additionally,  the  between  mission  maintenance  window 
check  forwards  aircraft  for  LRU  replacement  or  repair  if  the  RUL  prediction  is  within  the 
designated  maintenance  window  and  parts  are  in  stock.  Figure  12  depicts  the  simulation 
outcome  described  above,  growing  from  15.63  hours  to  35.40  hours  for  the  uncertainty 
factor  100  case  for  each  FA  limit.  Taken  over  the  15  years,  the  total  life  lost  ranges  from 
a  low  of  144506  hours  for  the  perfect  prognostics  condition  to  321236  hours  for  the  case 
where  uncertainty  factor  is  100  and  FA  limit  is  unlimited.  This  translates  to  36.67  years 
of  part  life  lost  for  the  latter  case.  The  mean  life  lost  for  each  FA  limit  case  is 
approximately  equal  at  each  point,  thus  they  are  collocated  in  the  figure.  This  results  from 
the  fact  that  while  the  ISHM  system  may  require  more  maintenance,  the  LRU 
components  are  only  replaced  as  required.  Of  note  is  the  max  total  life  lost  for  the  0  FA 
limit  case  is  207901,  occurring  at  an  uncertainty  factor  of  20.  The  total  life  lost  then 
continues  to  drop  off  as  the  uncertainty  factor  rises.  This  is  due  to  the  number  of  hours 
being  flown  by  the  aircraft  declining  as  the  uncertainty  factor  increases,  thus  not 
requiring  LRU  replacement  as  frequently.  The  lost  utilization  and  cost  implications  of 
this  figure  could  provide  justification  for  system  implementation.  Component  life  lost  in 
the  baseline  case  is  driven  by  the  time-based  preventive  maintenance  (PM)  cycle.  In  this 
research,  the  PM  cycle  is  set  at  400  hours  whereby  all  components  with  less  than  400 
hours  remaining,  by  time  accounting,  are  replaced,  yielding  a  mean  life  lost  of  376.49  and 
377.06  hours  for  the  (1,0)  and  (4,2)  cases  respectively. 
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Uncertainty  Factor 


Figure  12.  Mean  Life  Lost  per  Component 


False  alarms  and  maintenance  hours  are  important  when  detennining  cost,  support 
requirements,  and  system  confidence,  but  users,  whether  they  are  military  or  commercial, 
want  to  know  how  often  their  aircraft  are  available  and  when  tasked  if  they  can  complete 
the  mission.  Utilizing  Eq.  (4)  to  calculate  Ao,  Figure  13  shows  the  impact  of  uncertainty 
factor  and  FA  limit.  Operational  availability  drops  from  0.983  for  both  FA  limit  levels  at 
an  uncertainty  factor  of  0  to  0.754  for  the  uncertainty  factor  100,  FA  limit  0  case  and 
0.969  for  the  unlimited  FA  case.  The  increase  in  downtime  to  repair  the  ISHM  system  in 
the  0  FA  limit  case  is  the  driving  factor  in  the  decrease  in  Ao  over  the  uncertainty  levels. 
In  the  baseline  cases,  Ao  is  0.618  and  0.941  for  the  (1,0)  and  (4,2)  cases  respectively.  Ao 
is  low  in  the  (1,0)  case  again  for  the  NMCS  condition. 
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Figure  13.  Operational  Availability 


Sensor  and  Prognostic  Degradation  Results 

A  further  examination  of  the  impact  of  a  degrading  prognostics  capability  is 
examined  as  well.  Eq.  (2)  becomes: 


uncertainty  -  degredation  factor  +  In  (Part  RUL)  * 


uncertainty  factor 


(5) 


This  degradation  factor  places  an  additional  uncertainty  on  the  RUL  prediction 


given  as: 


degredation  factor  =  10  + 


Part  ISHM  timer 
growth  factor 


Where  growth  factor  is  either  50  or  200  to  provide  different  rates  of  degradation. 
Referring  to  Table  1,  it  is  shown  that  component  MTBF  is  bounded  between  250  and 
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1000  hours.  Therefore,  the  impact  on  RUL  uncertainty  could  grow  to  nearly  the 
component  life  in  the  case  of  part  P  if  left  unchecked.  The  Part  ISHM  timer  is  the 
accumulated  life  on  the  ISHM  components  associated  with  a  specific  component.  The 
timer  is  reset  upon  component  replacement  or  when  a  false  alarm  limit  is  reached  thereby 
initiating  maintenance  on  the  ISHM  system.  Degradation  factor  increases  as  a  function  of 
the  accumulated  time  on  the  Part  ISHM  timer.  Thus,  the  longer  the  ISHM  system  is  in 
operation,  the  higher  the  degradation  factor  becomes  adding  to  the  uncertainty  in  the 
system.  As  with  Eqs.  (1)  and  (2),  Eqs.  (5)  and  (6)  are  representative  equations  developed 
by  the  authors  to  portray  the  behavior  of  health  monitoring  systems. 

Including  the  degradation  factor  in  the  model  as  in  Eq.  (5)  shows  a  false  alann 
limit  may  be  useful  in  actual  aircraft  operation.  Fixing  the  error  factor  at  20,  towards  the 
lower  end  of  the  range,  an  exploration  of  the  impact  of  false  alann  limits  is  made.  The 
growth  factors  of  50  and  200,  utilized  in  Eq.  (6),  are  hereafter  referred  to  as  high  and  low 
respectively.  These  factors  correspond  to  a  growth  rate  of  20  and  5  per  hundred  hours  of 
accumulated  time  on  the  ISHM  system  respectively.  The  degradation  factor  adds 
additional  uncertainty  to  the  RUL  prediction  to  examine  the  effect  of  degrading  sensor  or 
prognostics  capability  through  use  of  the  aircraft.  In  the  analysis  of  degradation  factor, 

FA  limit  is  the  variable  of  change  and  is  varied  from  0  to  100. 

Examining  the  impact  of  FA  limit  on  mean  daily  flying  hours  for  the  squadron 
shows  that  the  0  FA  limit  case,  for  which  every  false  alarm  triggers  ISHM  maintenance, 
dramatically  reduces  the  flying  hours.  This  results  from  the  amount  of  maintenance 
required  on  the  ISHM  system  depleting  available  hours  to  fly  missions.  These  results  are 
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shown  in  Figure  14  and  indicate  that  the  low  degradation  growth  rate  reduces  the  flying 
hours  from  32.67  at  a  FA  limit  of  2  to  31.15  at  100.  In  contrast,  the  high  growth  rate 
drops  the  daily  hours  from  32.16  at  FA  limit  2  to  27.49  at  FA  limit  100.  The  difference  in 
the  magnitude  of  the  declines  lies  in  the  fact  that  the  high  degradation  rate  increases 
uncertainty  in  the  RUL  prediction,  thus  driving  false  alarm  occurrence  up.  That  is,  when 
the  FA  limit  is  0  and  there  is  a  false  alarm,  the  ISHM  system  is  always  repaired.  When 
the  FA  limit  increases  to  2,  this  allows  flights  to  continue  until  2  false  alarms  are 
incurred,  thus  allowing  increased  flying  hours  for  the  aircraft.  The  degradation  factor, 
slow  deterioration  of  prognostics  system,  accounts  for  the  remaining  decline  in  daily 
flying  hours.  This  results  from  compounded  error  in  the  system  increasing  as  the  time 
between  service  lengthens  due  to  the  FA  limit  being  raised. 
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Figure  14.  Daily  Hours  Flown 


Figure  8  shows  that  for  a  static  uncertainty  factor  of  20  the  MMH/FH  was  0. 165 
and  0.743  for  the  FA  limit  10000  and  0  cases  respectively.  Figure  15  below  shows  that 
the  high  degradation  rate  reaches  0.74  at  a  FA  limit  of  100  and  the  low  rate  0.378.  The 
graph  does  not  show  the  FA  limit  0  MMH/FH  data  of  2.361  for  the  low  and  2.468  for  the 
high  to  allow  better  visualization  of  the  remaining  data.  It  is  observed  in  Figure  16  that 
the  impact  of  the  high  growth  rate  greatly  increases  the  number  of  false  alanns,  thus 
increasing  the  maintenance  hours  required  per  aircraft  flight  hour  shown  in  Figure  15. 
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Figure  15.  Maintenance  Man  Hours  per  Flight  Hour 

As  previously  mentioned,  Figure  16  is  perhaps  the  best  indicator  of  the  impact  of 
degradation  growth  rates  on  aircraft  operations.  The  high  growth  rate  proves  true  to  its 
name  as  the  rate  of  increase  in  false  alarms  per  aircraft  per  year  remains  higher  than  the 
low  growth  rate  over  the  range  of  FA  limits.  The  number  of  false  alarms  increases  as  a 
result  of  the  degradation  factor  continually  increasing  as  the  ISHM  system  is  not  being 
maintained  at  the  shorter  intervals  a  lower  FA  limit  brings. 
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Figure  16.  False  Alarms  per  Aircraft  per  Year 


The  impact  of  the  increase  in  false  alarms,  and  thus  downtime,  is  a  decrease  in 
operational  availability,  Ao,  as  the  FA  limit  increases.  Shown  in  Figure  17,  the  Ao  trend 
follows  that  of  the  daily  flying  hours  and  inversely  the  trends  of  false  alarms  and 
MMH/FH.  Operational  availability  peaks  at  a  FA  limit  of  4  for  both  the  high  and  low 
growth  rates.  The  low  growth  rate  levels  off  around  0.96  at  FA  limit  60  while  the  high 
rate  continues  a  decline  to  0.93  at  FA  limit  100  without  leveling  off. 
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Figure  17.  Operational  Availability 


While  Figure  16  shows  the  increased  growth  in  number  of  false  alarms,  the  true 
utility  of  the  model  is  in  determining  the  “sweet  spot”  across  the  performance  curves. 

This  is  the  location  where  a  peak  or  trough  in  the  curves  indicates  performance  drops  off 
on  either  side  and  thus  this  set  of  factors  should  be  considered  for  system  design.  In  this 
paper,  examining  Figure  15  and  Figure  17  show  a  performance  drop  off  at  a  FA  limit  of 
4.  These  results  are  specific  to  the  set  of  inputs  used  in  the  model.  If  the  time  for 
inspection  of  a  failure  condition  or  to  repair  the  ISHM  system  were  changed,  the  potential 
for  a  different  outcome  in  FA  limits  exists.  Therein  lies  the  utility  of  the  model  in  being 
able  to  change  input  characteristics  and  policies  to  determine  system  level  performance 
metrics. 
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In  comparing  the  sensor  degradation  case  in  Figure  14  with  the  baseline  case, 
daily  flying  hours  remain  higher  than  the  baseline  case  across  the  FA  limit  range.  The 
MMH/FH  for  the  degradation  case  with  low  growth  rate  remains  below  that  of  the 
baseline  cases,  while  the  high  growth  rate  case  is  higher  than  the  baseline  cases  for  FA 
limits  above  40.  As  previously  discussed,  the  FA  limit  “sweet  spot”  in  this  model  is  4 
thus  MMH/FH  would  be  approximately  0.3  and  less  than  the  baseline  cases.  Comparing 
Ao  between  the  baseline  and  degradation  models  shows  that  around  the  4  FA  limit 
results,  the  degradation  cases  are  above  0.96  while  the  baseline  cases  are  0.618  and  0.941 
for  the  (1,0)  and  (4,2)  cases  respectively.  This  again  shows  the  ISHM  system  to  provide 
higher  performance.  Finally,  the  mission  reliability  for  the  baseline  cases  of  84.85%  is 
higher  than  the  ISHM  cases,  which  are  below  70%  at  the  4  FA  limit  case.  Across  the 
model  metrics  the  ISHM  case  with  degradation  tends  towards  higher  perfonnance  than 
the  baseline.  Depending  on  the  desired  performance  levels  desired  for  the  aircraft 
program  managers  are  left  to  weigh  the  performance  metrics. 

In  the  model  case  where  degradation  is  present,  for  the  uncertainty  factor  chosen 
it  is  generally  best  to  set  the  false  alarm  limit  low.  Programmatic  policy  of  cost, 
availability  and  reliability  will  drive  towards  the  selection  of  a  proper  limit.  Additionally, 
changes  to  degradation  factor,  i.e.,  ISHM  sensor  and  prognostic  characteristics,  and  RUL 
uncertainty,  prediction  algorithm  accuracy,  can  change  model  outcomes.  Cost  to 
implement  a  certain  health  monitoring  technology  on  the  aircraft  may  outweigh  the 
benefit  of  its  inclusion  if  it  drives  too  many  false  alarms  or  too  much  repair  time. 
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Absent  the  cost  impacts  of  manpower  and  component  replacement,  the  decision  as 
to  how  much  uncertainty  in  prognostics  is  an  easier  proposition.  It  is  shown  in  the  model 
with  no  degradation  that  as  the  RUL  uncertainty  increases,  most  perfonnance 
characteristics  are  adversely  impacted.  The  comparison  of  baseline  to  ISHM  cases  shows 
the  potential  advantages  implementation  of  health  monitoring  and  condition  based 
maintenance.  The  test  for  program  managers  then  becomes  selecting  the  appropriate 
system  characteristics  to  meet  overall  aircraft  fleet  performance  and  cost  metrics. 

Conclusion 

This  research  shows  employment  of  an  ISHM  system  supporting  CBM  can 
produce  system  perfonnance  greater  than  baseline  systems.  The  main  contribution  of  this 
effort  is  as  a  simulation  tool  to  compare  sensing  options  and  examine  their  impact  on 
desired  perfonnance  factors.  The  ability  to  input  ISHM  system  and  aircraft  characteristics 
and  investigate  alternative  approaches  to  monitoring  and  maintenance  makes  this  tool 
useful  in  program  decisions  on  whether  or  not  to  implement  monitoring  techniques. 

While  determining  causes  of  system  uncertainty  is  outside  the  scope  of  this  research, 
quantifying  the  impact  of  the  uncertainty  is  demonstrated.  As  a  system  designer  it  is 
important  to  note,  as  this  research  shows,  the  amount  of  uncertainty  in  your  system, 
particularly  in  the  prognostics.  This  uncertainty  could  be  mitigated  with  better  sensors, 
techniques  or  processing  algorithms.  Further,  the  designer  should  seek  to  minimize  either 
the  number  or  false  alanns  the  prognostic  system  produces  or  set  an  appropriate  limit  on 
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false  alarms  to  minimize  the  impact  of  additional  inspection  time  to  adjudicate  system 
condition. 

As  cost  is  not  included  in  this  work  making  a  true  comparison  among  options  is 
difficult.  A  program  manager  must  weigh  the  technology  costs  to  achieve  the 
performance  observed  in  the  model  and  compare  those  with  system  objectives.  This  task 
becomes  easier  if  these  variables  can  be  explored  across  a  range  of  scenarios  as  this 
research  provides. 

Future  work  in  this  research  will  explore  the  impact  of  cost,  supply  factors  and 
manpower  requirements. 
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IV.  Integrated  System  Health  Monitoring  Impact  on  Non-Repairable  Component 


Supply  Methods 


Chapter  Overview 

From  on-board  automotive  diagnostics  to  real-time  aircraft  state  of  health,  the 
implementation  of  health  monitoring  and  management  systems  are  an  increasing  trend. 
This  research  analyzes  the  impact  of  a  health  monitoring  system  on  a  squadron  of 
aircraft.  Flight,  maintenance  and  logistics  operations  are  stochastically  modeled  to 
determine  the  impact  of  program  decisions  on  supply  metrics.  An  Arena  8  discrete  event 
simulation  is  utilized  to  conduct  this  research  on  20  components  on  each  of  the  12  aircraft 
modeled.  Costs  and  availability  are  recorded  for  comparison  across  three  sparing 
scenarios  to  include  economic  order  quantity  for  baseline  and  health  monitoring  cases 
and  a  just-in-time  health  monitoring  set  of  simulations.  Finally,  the  different 
methodologies  are  compared  and  discussed  as  a  trade-space  for  programmatic  decisions. 

Introduction 

The  development  of  vehicle  health  monitoring  systems  enables  a  focus  on 
condition  based  maintenance  in  lieu  of  time-based  preventive  maintenance.  With 
increased  knowledge  about  vehicle  systems  gained  through  data  collected  by  health 
monitoring  systems,  logistics  operations  too  should  be  considered.  In  (Hess  &  Fila,  2002; 
Hess,  Calvello,  &  Dabney,  2004)  Hess  et  al.  discuss  the  potential  benefits  of  informed 
logistic  systems  enabled  by  prognostics  and  health  management  systems,  in  particular  the 
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autonomic  logistics  infonnation  system  (ALIS)  of  the  F-35  tighter.  With  the  development 
and  fielding  of  health  monitoring  systems  on  a  range  of  vehicles,  taking  full  advantage  of 
the  infonnation  collected  is  an  evolving  task. 

Health  monitoring  efforts  can  be  integrated  with  the  logistics  chain  to  optimize 
maintenance  actions  with  supply  requirements.  The  future  goal  of  integrated  systems 
health  management  and  condition  based  maintenance  (ISHM/CBM)  integration  is  to 
yield  operations  and  sustainment  (O&S)  savings  (Carnero  Moya,  2004). 

The  General  Accountability  Office  (GAO)  has  conducted  several  audits  of  the 
Department  of  Defense  (DoD),  with  findings  ranging  from  required  items  routinely 
missing  from  inventories  to  enonnous  on-hand  stock.  Additionally,  they  found  unneeded 
parts  on  order  accounting  for  billions  of  dollars  in  unneeded  parts  and  operational  rates 
well  below  goals  (Government  Accountability  Office,  2001;  Government  Accountability 
Office,  2007).  Reasons  for  parts  shortages  range  from  inadequate  forecasting,  repair  and 
manufacturing  issues,  poor  replacement  part  reliability,  and  contracting  problems.  In  one 
extreme  example,  demand  for  an  engine  bolt  was  projected  at  828  units  when  actual 
requirements  were  over  12,000  in  a  single  quarter  (Government  Accountability  Office, 
2001).  This  issue  was  still  present  in  2010  when  the  Logistics  Management  Institute 
conducted  a  review  of  DoD  logistics  citing  “inaccurate  demand  forecasting  as  a  primary 
cause  for  the  military  services’  inability  to  align  inventory  levels  with  current  demands” 
(Atchley  et  ah,  2010,  p.  2-5).  Implementing  ISHM/CBM  systems  on  a  broad  range  across 
the  services  would  increase  the  forecasting  capability  and  provide  near  real  time  insights 
into  requirements,  both  existing  and  in  the  near  future.  Between  2002  and  2005,  Air 
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Force  spares  shortages  were  between  6  and  8  percent  of  total  requirements,  averaging 
over  $1  billion  in  shortfalls  (Government  Accountability  Office,  2007). 

A  shortage  of  spares  is  not  the  only  issue  facing  the  Air  Force.  As  cited  by  the 
GAO,  “the  value  of  Air  Force  on-order  inventory  not  needed  to  support  required 
inventory  levels...  represents]  an  average  of  52%  ($1.3  billion)  of  its  on-order  inventory” 
(Government  Accountability  Office,  2007,  p.  9).  Between  2002  and  2005,  65%  of  on- 
hand  inventory — amounting  to  $18.7  billion — was  not  required  for  projected  use  rates. 
Further  compounding  the  unneeded  on-hand  inventory  is  that  some  items  have  a  shelf  life 
that  requires  disposal  or  refurbishment  after  a  set  amount  of  time.  The  prognostics 
capability  of  ISHM  systems  can  increase  the  reliability  of  spares  forecasting  and, 
depending  on  projected  use  rates,  keep  sparing  purchases  within  DoD  budget  timelines. 

The  remainder  of  this  paper  is  organized  as  follows.  Section  2  reviews  relevant 
literature  on  health  monitoring  and  supply  and  discusses  supply  concepts.  Section  3 
introduces  the  model  formulation  and  measures  of  merit.  The  results  for  the  different 
methodologies  utilized  in  this  research  are  provided  in  Section  4.  Finally,  section  5 
provides  a  conclusion  for  the  work  and  discusses  future  efforts. 

Background 

This  section  reviews  relevant  literature  and  discusses  concepts  behind  model 
formulation. 
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Cost  Accounting 

The  cost  of  ordering  replacement  parts  can  be  decomposed  into  two  elements, 
administrative  cost  and  part  cost  (Sherbrooke,  2004). 

OC  =  K  +  P*Q  (7) 

where  OC  is  the  order  cost,  K  is  the  fixed  administrative  cost  per  order,  P  is  the 
component  cost,  and  Q  is  the  order  quantity. 

Once  the  ordered  spares  are  delivered  to  the  customer,  holding  costs  are  then 
accrued.  These  costs  are  related  to  the  storage,  storage  buildings  and  upkeep,  spares 
inventorying  and  maintenance,  insurance,  etc.  These  costs  are  generally  calculated  as  a 
percentage  of  the  per  unit  part  cost  and  are  determined  either  over  discrete  time  periods 
or  continuously  as  in  Eq.  (8).  The  cost  of  deterioration  is  also  included  in  holding  cost, 
but  is  not  addressed  in  this  research.  Shah,  Soni,  and  Patel  (Shah,  Soni,  &  Patel,  2013) 
propose  a  method  to  include  the  deteriorating  spares  cost  in  the  holding  cost  equation. 

T 

HC  =  \H*SL{t)dt  (8) 

o 

where  HC  is  the  holding  cost,  H  is  the  part  holding  cost  per  time  and  SL(t)  is  the  stock 
level  held  in  inventory  at  time  t  (Rodrigues  &  Yoneyama,  2012). 

In  the  commercial  arena,  there  is  a  tangible  cost  for  not  having  needed  stock  on 
hand  when  required,  known  as  a  stockout  cost,  such  as  lost  revenue  for  a  flight  or  halting 
of  production.  In  the  government  or  military  operating  environment,  where  revenue  is  not 
a  consideration,  the  impact  is  more  likely  in  the  form  of  the  system  being  labeled  non¬ 
mission  capable  for  supply  (NMCS).  Depending  on  the  spare  parts  strategy  used  for  the 
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system,  a  NMCS  could  result  in  contractual  penalties  to  suppliers  (Anderson,  Fitzsimons, 
&  Simester,  2006;  Kennedy,  Wayne  Patterson,  &  Fredendall,  2002).  Company 
reputations  as  well  as  relations  with  customers  are  also  indirect  costs  of  stockout 
conditions  (Rodrigues  &  Yoneyama,  2012).  Sustained  high  levels  of  NMCS  aircraft  may 
also  require  a  larger  fleet  to  meet  mission  requirements. 

Economic  Order  Quantity 

The  concept  of  an  optimal  or  economic  order  quantity  (EOQ)  was  introduced  by 
Harris  in  1913  (Harris,  1990).  Shown  in  Eq.  (9),  this  model  is  concerned  with  a  reorder 
point,  R,  and  an  order  quantity,  Q,  yielding  an  (R,  Q)  model.  The  reorder  point  is 
determined  based  on  the  desired  safety  stock  and  order  lead  time.  The  economic  order 
quantity  is  determined  by, 

Q  =  V2  *D*K/H  (9) 

where  Q  is  the  quantity  of  spare  parts  to  be  purchased  when  a  new  order  is  placed,  D  is 
the  average  demand  per  unit  of  time,  K  is  the  administrative  cost  of  placing  an  order,  and 
H  is  the  holding  cost  per  unit  per  unit  of  time  held  in  inventory  (Sherbrooke,  2004). 

While  a  traditional  (R,  Q)  supply  methodology  augmented  by  ISHM  projections 
could  work,  as  shown  by  Rodriguez  and  Yoneyama  (2012),  a  just-in-time  approach  may 
work  as  well. 

Prior  Models 

Kahlert,  Giljohanan  and  Klingauf  (2014)  use  a  MATLAB  discrete  event 
simulation  to  analyze  a  single  commercial  component  with  100%  unscheduled 
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maintenance.  They  evaluate  health  monitoring  system  perfonnance  and  conclude  a 
potential  annual  savings  of  approximately  20%  is  possible. 

Rebulanan  models  the  F-35  ALIS  system  with  a  discrete  event  simulation 
containing  LRUs,  communication  system,  supply,  and  maintenance  systems  (Rebulanan, 
2000).  Rebulanan  evaluates  model  performance  with  aircraft  availability,  mission  capable 
and  non-mission  capable  rates.  The  simulation  shows  sensitivity  of  supply  lead  time  to 
the  detection  lead  time  and  stock  levels. 

Rodrigues  and  Yoneyama  (2012;  2013)  examine  the  impact  of  ISHM  prognostics 
on  stock  levels  for  both  repairable  and  non-repairable  systems.  These  models  are 
compared  with  a  conventional  supply  system.  These  studies  both  indicate  potential  cost 
savings  for  a  prognostic  system  versus  a  conventional  supply  process.  The  non-repairable 
study  includes  only  one  item  with  a  fixed  supply  lead  time.  This  limitation  leaves  the 
interaction  of  multiple  components  and  variable  lead  times  in  question. 

Methodology 

The  research  methodology  used  in  this  work  builds  upon  Vandawaker,  Jacques, 
and  Freels  (2015).  The  Arena®  discrete  event  simulation  processes  a  squadron  of  12 
aircraft  through  daily  operations  to  include:  flying,  take-offs  and  landings,  inspections, 
maintenance,  supply  and  health  monitoring  activities.  Each  simulation  takes  the  aircraft 
through  15  years  of  operation  and  consists  of  100  runs.  The  architecture  for  the  ISHM 
model  processes  is  found  in  Figure  6.  This  figure  depicts  the  flow  of  information  and 
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aircraft  through  system  operations.  System  processes  include  flight  operations,  ISHM 
diagnostics  and  prognostics,  CBM  order  processing,  and  maintenance  and  supply  actions. 


ISHM  System  CBM  System  Maintenance  Supply 


A  baseline  model  is  run  as  a  comparison  with  time -based  preventive  maintenance 
and  (R,  Q)  stocking  policy.  An  ISHM  model  is  also  run  with  an  (R,  Q)  policy  based  on 
Rodrigues  and  Yoneyama  (2012)  where  the  reorder  point  is  adjusted  based  on  need 
predicted  by  the  ISHM  system.  These  are  both  compared  to  a  supply  system  operating 
nearer  to  a  just-in-time  delivery  program. 

Just-in-Time  Logistics 

Hess  and  Fila  (  2002)  discuss  the  ability  of  health  monitoring  systems  and 


automated  logistics  to  enable  just-in-time  (JIT)  inventory  methods.  There  are,  however, 
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conflicting  views  regarding  the  efficacy  of  just-in-time  sparing.  Bragalia,  Grassi,  and 
Montanari  (  2004)  cite  just-in-time  sparing  as  “the  most  desirable  approach,”  but  stress 
good  integration  between  supplier  and  customer.  Kennedy,  Patterson,  and  Fredendall 
(2002)  cite  JIT  sparing  as  a  desirable  option  for  predictable  demand  items,  while  also 
stating  that  JIT  may  not  be  a  good  option  when  demand  is  unpredictable.  Huiskonen 
(2001)  also  suggests  a  strong  relationship  between  the  supplier  and  customer  is  necessary 
to  ensure  efficient  and  effective  inventory  management  if  JIT  parts  management  is 
chosen. 

In  this  work,  ensuring  adequate  notification  time  prior  to  component  need  is 
accomplished  through  a  daily  analysis  of  projected  component  needs.  This  lead  time  is 
shown  in  Eq.  (10)  and  includes  factors  to  mitigate  the  risk  of  a  stockout  condition  prior  to 
component  need  by  maintenance, 

SF  +  MW  +  JIT  factor  > Part  _RUL  (10) 

where  SF  is  the  component  safety  factor,  MW  is  the  maintenance  window  to  make 
preemptive  repairs,  JIT  factor  accounts  for  supply  lead  time,  and  Part  RUL  is  the 
remaining  useful  life  of  the  part  in  question. 

This  research  used  the  methodology  above  for  just-in-time  supply  with  order 
windows  of  1,  3  and  5  days  across  the  simulation  runs. 

The  modeling  assumptions  for  this  work  are  as  follows: 

•  Components  are  non-repairable; 

•  Components  do  not  degrade  on  shelf; 

•  Cost  of  money  is  constant  across  simulation  time; 

•  No  change  in  delivery  time  distribution  over  simulation; 
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•  No  cost  penalty  for  backorder; 

•  Cost  of  supplies  remains  constant; 

•  There  is  no  discount  for  large  quantity  order; 

•  (R,  Q)  model  unique  for  each  part; 

•  If  a  stockout  condition  occurs  before  order  is  received,  aircraft  is  NMCS  until 
part(s)  arrive; 

•  Each  order  incurs  a  fixed  administrative  cost,  K,  regardless  of  ordering 
process; 

•  A  holding  cost,  HC,  is  incurred  annually  for  each  unit  in  inventory; 

•  Enough  maintenance  personnel  are  available  to  complete  tasks  without  delay. 

Model  Components  and  Architecture 

Each  aircraft  has  20  components,  denoted  A-T,  with  failure  times  randomly 
sampled  from  an  exponential  distribution,  with  a  given  mean  time  between  failure 

-X/ 

(MTBF),  and  with  probability  distribution  function:  /(A)  =  x/pe  ''  ,  for  x  >  0  . 

In  the  baseline  case,  unless  the  part  is  scheduled  for  preventive  maintenance,  the 
condition  is  not  known;  thus  the  need  for  repair  or  replacement  is  unanticipated  and  the 
aircraft  continues  nonnal  mission  operations.  Maintenance  actions  are  performed  in 
parallel  on  each  aircraft.  All  part  inspection  times  are  in  minutes  and  are  triangularly 
distributed  (20,  30,  45).  LRU  replacement  times  are  also  in  minutes  and  triangularly 
distributed  (60,  90,  240). 

Supplies  are  input  into  the  model  at  an  initial  stock  level  and  a  reorder  point.  In 

the  baseline  case,  the  reorder  point  and  economic  order  quantity  are  fixed  throughout  the 

simulation.  The  baseline  reorder  point  is  two  for  all  components,  leaving  a  small  safety 

stock  to  lessen  the  probability  of  running  out  of  spares  before  the  next  need.  In  the  ISHM 

94 


EOQ  case,  the  order  quantity,  Q,  is  fixed,  but  the  reorder  point,  R,  varies  based  on 
predicted  need  from  the  ISHM  system  prognostics.  In  the  just-in-time  cases,  an  initial 
stock  level  of  five  is  assigned  and  the  ISHM  prognostics  decide  when  and  what  quantities 
of  parts  are  ordered.  The  ordering  system  for  the  just-in-time  cases  scans  all  aircraft  daily 
and  determines  the  required  number  of  each  component  over  a  certain  lead  time  to  need, 
shown  in  Eq.  (10),  and  orders  are  made  at  a  set  interval  of  days  to  meet  the  demand.  EOQ 
is  not  considered  in  the  just-in-time  cases.  Supply  lead  time  is  log-normally  distributed 
(5,  2)  days  between  delivery  and  receipt  for  all  parts. 

The  remaining  useful  life  (RUL)  prognostic  technique  used  in  this  research  takes 
into  account  sensor  degradation  and  inherent  uncertainty  in  the  system  as  described  in 
Vandawaker  (Vandawaker,  Jacques,  &  Freels,  2015).  The  health  monitoring  system 
orders  spares  to  meet  demand  as  described  in  section  2.  If  a  component  is  in  a  stockout 
condition,  the  aircraft  becomes  non-mission  capable  for  supply  awaiting  needed  part(s). 

Evaluation  Parameters 

Two  categories  of  metrics  are  applicable  to  the  results  of  this  research:  availability 
and  cost. 

Operational  availability  (Ao)  includes  logistics,  supply  and  administrative  delays 
to  preventive  and  corrective  maintenance  for  the  system  resulting  in  the  mean  downtime 
for  the  system.  Operational  availability  is  the  system  availability  the  user  of  a  system 
realizes  (ReliaSoft,  2007).  Mathematically,  operational  availability  is: 

.  Uptime  MTBM 

A  = - - - = -  (11) 

Uptime  +  Downtime  MTBM  +  MMT  +  MLDT 
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Where:  MTBM  is  the  mean  time  between  maintenance,  MLDT  is  the  mean  logistics 
delay  time,  and  MMT  is  the  mean  maintenance  time. 

Non-mission  capable  supply  (NMCS)  is  the  time  between  component  need  by 
maintenance  and  when  the  part  arrives  in  maintenance.  ISHM/CBM’s  prognostic  ability 
can  predict  impending  component  failure  and  establish  sparing  requirements  in  advance 
of  maintenance  requirements,  ordering  spares  if  stock  levels  are  inadequate.  The  baseline 
process  relies  on  predicted  failure  rates  and  providing  stock  levels  to  meet  anticipated 
demands.  This  process  increases  the  logistic  footprint  by  requiring  storage  facilities  for 
materiel  that  may  not  be  needed  for  upwards  of  a  year.  Managing  these  spares  and  the 
facility  requires  additional  resources,  manpower,  and  money.  Holding  cost  is  another  area 
of  interest  as  it  drives  facility  and  manpower  requirements. 

Results 

The  data  presented  here  shows  the  impacts  of  part  ordering  based  on  projected 
need  of  all  aircraft  over  the  duration  in  Eq.  (10).  Additionally,  EOQ  models  with  both 
ISHM  and  baseline  architectures  are  provided  for  comparison. 

Figure  19  shows  the  impact  of  the  supply  ordering  methodology.  It  is  observed 
that  as  the  number  of  days  between  JIT  orders  increases,  the  yearly  average  NMCS  hours 
per  aircraft  increases  as  well.  The  increase  is  attributed  to  variability  in  aircraft  usage  and 
supply  delivery  time  over  the  period  between  ordering  and  actual  receipt  of  the  parts.  The 
baseline  case  with  a  (R,  Q)  ordering  scheme  for  spare  parts  yields  a  yearly  NMCS  rate 
per  aircraft  of  57  hours.  This  baseline  result  represents  a  26%  lower  rate  than  ordering 
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parts  every  5  days,  but  a  90%  increase  over  the  3  day  ordering  schedule  and  an  850% 
increase  over  daily  ordering.  Notably,  the  ISHM  EOQ  case  has  the  lowest  annual  mean 
NMCS  time,  less  than  4  hours  per  aircraft.  The  high  supply  availability  is  attributed  to  the 
on-hand  stock  kept  versus  the  JIT  method  and  the  forward  looking  capability  afforded  by 
the  ISHM  prognostics. 


Order  Frequency  (days) 

Figure  19.  NMCS  Hours  per  Aircraft  per  Year 

While  it  may  be  desirable  to  choose  a  methodology  with  the  lowest  NMCS  rate,  it 
does  not  come  without  an  accompanying  cost.  Shown  in  Figure  20  below,  increased 
holding  cost  is  a  trade-off  of  a  low  NMCS  rate.  Holding  cost  decreases  as  the  order 
frequency  increases.  This  increase  is  a  direct  result  of  spares  being  in  stock  for  a  longer 
duration  awaiting  installation  on  the  aircraft.  In  the  baseline  case,  mean  yearly  holding 
costs  are  $106K.  As  shown  in  Figure  20,  this  represents  a  104%  increase  over  daily 


ordering,  a  171%  increase  over  3  day  order  frequency,  and  a  216%  increase  over  5  day 
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ordering.  While  the  holding  costs  may  not  appear  to  be  a  large  value  in  absolute  terms,  it 
must  be  noted  that  these  costs  are  only  for  20  components  in  a  fleet  of  12  aircraft  at  one 
operating  location.  When  scaled  up  to  thousands  of  parts  and  hundreds  of  aircraft,  and 
operating  at  locations  around  the  world,  these  costs  can  be  sizeable. 
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1  3  5  Baseline  EOQ  ISHM  EOQ 

Order  Frequency  (days) 


Figure  20.  Holding  Cost 

As  discussed  in  section  2,  administrative  order  costs  are  those  costs  associated 
with  the  processing,  shipping  and  handling  of  an  order  above  the  cost  for  the  actual 
part(s).  As  shown  in  Figure  21,  the  administrative  costs  for  the  just-in-time  delivery 
method  are  considerably  higher  than  for  the  EOQ  schemes  for  both  the  baseline  and 
ISHM  cases.  This  results  from  the  fact  that  orders  are  being  placed  much  more  frequently 
to  meet  demand  as  the  stock  level  for  the  JIT  case  is  kept  to  only  what  is  projected  over 
the  next  several  days  of  flying.  The  EOQ  cases  take  into  account  economic  factors  of  the 
supply  process  to  keep  overall  cost  low,  at  a  higher  overall  holding  cost  and  stock  level. 
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If  JIT  administrative  costs  can  be  brought  down  through  contracts  or  arrangements  with 
suppliers,  then  the  JIT  system  could  be  more  competitive  with  the  EOQ  ordering  scheme. 


600 


1  3  5  Baseline  EOQ  ISHM  EOQ 

Order  Frequency  (days) 


Figure  21.  Administrative  Cost 

Figure  22  represents  the  total  annual  supply  cost  for  each  scenario;  this  includes, 
holding  costs,  administrative  costs,  and  part  costs.  Observed  in  Figure  21,  administrative 
ordering  costs  account  for  the  majority  of  the  total  supply  cost  differences.  While  the 
baseline  cost  has  the  smallest  annual  cost,  it  is  only  part  of  the  picture.  When 
maintenance  costs  are  included  in  Figure  23,  the  baseline  case  is  no  longer  the  lowest 
cost.  Further,  when  considering  Figure  24  and  Figure  25,  the  baseline  case  has  the  lowest 
operational  availably  and  six  fewer  daily  flying  hours  across  the  12  aircraft.  Six  flying 
hours  per  day  is  equivalent  to  requiring  two  fewer  aircraft  in  the  ISHM  cases  for 
comparison. 
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2.5 


1  3  5  Baseline  EOQ  ISHM  EOQ 

Order  Frequency  (days) 


Figure  22.  Total  Supply  Cost 


Summing  total  supply  costs  along  with  the  direct  maintenance  costs,  actual 
maintenance  personnel  inspecting  or  replacing  parts  on  aircraft,  yields  the  total  cost 
shown  in  Figure  23.  This  comparison  shows  a  spread  of  approximately  $500,000 
annually  across  the  model  cases.  It  is  noted  that  the  maintenance  cost  for  the  baseline 
case  is  2.5  times  higher,  resulting  from  fixed  timed  based  preventive  replacement  versus 
the  condition  based  replacement  under  the  ISHM  logic. 
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Figure  23.  Total  Cost 


Operational  availability  is  of  concern  for  most  systems,  and  aircraft  are  no 
exception,  whether  commercial  carriers  or  military  transports.  If  aircraft  aren’t  available, 
the  mission  either  doesn’t  get  accomplished  or  it  is  delayed,  costing  money  and/or 
reputation.  Figure  24  shows  the  impact  of  the  supply  methodologies  in  this  research.  The 
baseline  case  shows  the  lowest  operational  availability  resulting  primarily,  as  mentioned 
above,  from  additional  time -based  maintenance  and  inspections.  A0  for  the  ISFIM  cases 
remain  between  0.97  and  0.98  for  all  the  scenarios,  resulting  from  fewer  required 
maintenance  activities.  Fewer  maintenance  activities  also  means  more  time  to  conduct 
flying  operations.  Shown  in  Figure  25,  the  daily  flying  hours  for  the  ISHM  cases  are 
higher  than  for  the  baseline  case. 
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Order  Frequency  (days) 

Figure  24.  Operational  Availability 

While  cost  is  a  chief  consideration  of  the  operation  of  most  systems,  it  is  not  the 
only  concern  or  measure  of  merit.  Selection  of  any  system  requires  a  balance  among 
often  competing  objectives.  In  this  research,  cost  and  availability  or  flying  hours  are 
somewhat  diametric  parameters.  While  supply  costs  for  all  ISFIM  cases  are  higher  than 
the  baseline,  it  is  not  the  only  consideration.  Management  must  weigh  the  savings 
potential  for  the  supply  methodology  chosen  with  the  need  fewer  aircraft  to  accomplish 
the  same  mission.  So  while  status  quo  of  the  baseline  system  shows  a  savings  of  17% 
annually,  the  ISHM  cases  can  provide  better  A0  and  equivalent  flying  hours  with  two 
fewer  aircraft.  If  the  cost  savings  of  removing  two  aircraft  and  the  cost  to  instrument  and 
implement  ISHM  across  the  remaining  aircraft  is  less  than  the  baseline  case,  the 
economic  case  is  made  to  implement  ISHM.  Additionally,  if  the  two  additional  aircraft 


102 


are  kept  as  well,  new  missions  could  be  accepted  to  increase  productivity  and  usefulness 
of  the  systems. 


1  3  5  Baseline  EOQ  ISHM  EOQ 

Order  Frequency  (days) 


Figure  25.  Daily  Flying  Hours 


Conclusion 

This  model  is  meant  as  a  planning  and  design  tool  to  study  the  impact  of  ISHM 
capabilities,  maintenance  processes,  and  spares  management  considerations.  Weighing 
the  impacts  of  simulation  outputs  and  management  philosophies  with  cost  and 
performance  objectives  is  left  to  program  managers  to  determine  the  appropriate  level  of 
service  required. 

The  cases  laid  out  above  provide  an  example  of  the  capability  of  integrated 
system  health  management  enabling  condition  based  maintenance  to  provide  a 
competitive  cost  to  traditional  aircraft  operations.  The  tool  developed  by  the  authors 
provides  insight  into  the  trade-space  of  health  monitoring  requirements,  sparing  decisions 
and  maintenance  operations.  If  the  administrative  cost  of  just-in-time  sparing  can  be 
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reduced,  it  is  possible  to  compete  with  a  modified  economic  order  quantity  sparing 
system.  Additionally,  these  are  fictional  cases  and  as  discussed  by  numerous  authors,  the 
truth  likely  lies  somewhere  in  between  (Braglia  et  ah,  2004;  Camero  Moya,  2004; 
Huiskonen,  2001;  Kennedy  et  ah,  2002;  Wheatley,  Gzara,  &  Jewkes,  2015). 

The  results  of  this  research  show  that  switching  from  a  baseline  EOQ  system  to 
and  ISHM  EOQ  system  can  provide  cost  savings,  absent  the  initial  investment  of  the 
health  management  system.  The  tool  and  simulation  outputs  are  then  useful  in 
determining  if  the  long-term  performance  increases  of  the  ISHM  system  balance  out  the 
additional  upfront  cost  of  implementation. 

The  next  step  in  this  research  is  a  study  of  maintenance  manning  decisions  based 
on  health  monitoring  inputs.  Further  study  of  this  area  of  interest  should  also  include 
repairable  systems  and  their  impact  on  the  supply  chain  and  manning. 
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V.  An  Examination  of  System  Health  Monitoring  Impact  Non-Repairable 


Component  Maintenance  Manning 


Chapter  Overview 

Reductions  in  operating  budgets  are  forcing  many  companies  and  militaries  to 
consider  reducing  manpower,  in  particular  maintenance  personnel.  Combined  with  longer 
service  lives  for  aircraft  and  other  systems,  maintenance  and  operations  processes  must 
be  reconsidered.  One  research  and  development  method  to  support  the  above 
considerations  is  health  monitoring  systems.  The  majority  of  published  research  efforts 
focus  on  health  monitoring  techniques  and  technologies,  leaving  others  to  determine  the 
maintenance  and  logistics  impact  on  the  systems.  This  research  utilizes  an  Arena® 
discrete  event  simulation  to  examine  a  squadron  of  12  military  cargo  aircraft,  monitoring 
20  components  on  each,  equipped  with  health  monitoring  systems.  The  impact  of  health 
monitoring  on  maintenance  manning  decisions  and  system  performance  is  studied.  Data 
presented  for  numerous  manning  levels  show  potential  cost  and  performance  trade-offs. 
Additionally,  a  case  study  exploring  the  impact  of  two  limited  manning  levels  is 
presented.  The  value  of  this  work  is  in  showing  the  ability  of  health  monitoring  systems 
to  affect  condition  based  maintenance  decisions.  Additionally,  the  development  of  trade- 
spaces  within  operating  environments  is  demonstrated  along  with  the  ability  to  conduct 
cost  benefit  analyses. 
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Introduction 


Maintenance  has  evolved  over  time  from  a  “fix  it  when  it  breaks”  policy  towards 
the  increasingly  popular  condition  based  maintenance  programs.  Rising  costs,  longer 
service  lives  and  reduced  manpower  has  driven  a  proactive  approach  to  maintaining 
systems.  The  reactive  or  corrective  maintenance  approach  of  many  systems  forces  either 
a  costly  spares  stockpile  to  prepare  for  all  possible  failures  or  waiting  for  replacement 
parts  to  arrive  resulting  in  zero  operational  availability  (Amari  et  al.,  2006).  One  of  the 
first  advancements  in  maintenance  practice  was  to  establish  regular  inspection  and 
preventive  maintenance  (PM)  intervals.  This  technique  analyzed  various  fonns  of  system 
performance  data  to  detennine  appropriate  times  to  inspect  and  replace  components 
(Walls  et  al.,  1999).  This  PM  approach  benefits  in  reduced  catastrophic  failures  at  the 
expense  of  more  maintenance  cycles  and  higher  maintenance  cost  (Deputy  Under 
Secretary  of  Defense  for  Logistics  and  Materiel  Readiness,  May,  2008).  Unanticipated 
failures  still  occur  outside  the  preprogrammed  maintenance  windows  and  must  be  taken 
into  account.  Further,  PM  subjects  the  system  to  unnecessary  “repair”  based  on  the 
required  schedule  for  the  system.  The  unneeded  maintenance  adds  extra  expense  to  the 
system  since  the  component  may  have  had  remaining  useful  life.  Moreover,  the 
probability  of  failure  can  increase  as  damage  often  occurs  during  maintenance  actions. 

Another  approach  to  system  maintenance  is  the  concept  of  selective  maintenance, 
whereby  a  subset  of  actions  are  performed  from  a  group  of  proposed  maintenance  tasks 
(Iyoob  et  al.,  2006).  As  Iyoob,  Cassady,  and  Pohl  (2006)  point  out,  a  majority  of  studies 
into  maintenance  practices  ignore  budget  limitations,  manpower  or  time  constraints, 
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which  is  where  selective  maintenance  can  assist  in  detennining  what  actions  should  be 
taken  to  maximize  the  available  resources.  They  present  logic  for  determining 
maintenance  actions  when  “it  may  be  impossible  to  make  all  possible  repairs  before  the 
next  mission.”  The  selective  maintenance  process  can  affect  all  maintenance  approaches, 
but  repair  driven  by  material  state  afforded  by  condition  based  maintenance  (CBM) 
allows  for  the  management  of  downtime  and  resources.  CBM  and  other  predictive 
maintenance  programs  have  further  evolved  from  preventive  strategies  to  actively  or 
passively  analyzing  system  health  and  forecasting  remaining  life. 

Much  research  on  health  monitoring  systems  is  focused  on  obtaining  system  or 
component  states  of  health  and  predicting  remaining  useful  life  (RUL).  These  works 
generally  provide  estimates  of  savings  in  maintenance  based  upon  the  ability  to  utilize 
parts  longer  before  replacement  or  through  the  elimination  or  reduction  of  inspections  to 
determine  materiel  condition: 

•  40%  for  vehicle  maintenance  (Walls  et  ah,  1999) 

•  30%  to  50%  for  fuselage  panels  (Pattabhiraman  et  ah,  2010) 

•  10%  electrical  components  (Scanff  et  ah,  2007) 

•  50-80%  for  the  Boeing  777  (Gorinevsky  et  ah,  2005). 

Lacking  in  these  studies  is  the  impact  an  integrated  system  health  management 
(ISHM)  approach  could  have  on  maintenance  manning  decisions.  If  inspections  are  no 
longer  needed  or  occur  on  a  less  frequent  basis,  are  the  personnel  devoted  to  time-based 
inspections  and  repair  needed  at  all?  What  needs  to  be  addressed  to  fully  understand  the 
impact  of  these  works  are  potential  savings  through  a  reduction  or  realignment  of 
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maintenance  manpower.  This  paper  lays  out  a  study  of  the  implications  of  integrated 
systems  health  management  on  maintenance  manning  levels  and  the  associated  cost  and 
aircraft  availability  impacts. 

Background 

Condition  based  maintenance,  when  appropriately  applied,  can  reduce  the  life 
cycle  cost  of  a  system.  Ellis  (2008)  argues  that  cost-effective  systems  monitoring  allows 
repair  actions  based  on  system  condition  rather  than  costly  time -based  maintenance. 
Additionally,  maintenance  can  be  forecast  for  completion  at  a  time,  likely  between 
projected  flying  sorties,  that  minimizes  impact  on  the  operational  mission  of  the  system. 
Under  a  CBM  paradigm,  interim  time-based  inspections,  required  under  the  baseline  PM 
approach  are  forgone  in  lieu  of  a  continuous  analysis  of  the  aircraft  via  the  ISHM  system. 

Maintenance  Manning 

While  this  work  does  not  propose  a  new  technique  to  optimize  maintenance 
manning,  it  does  seek  to  utilize  tools  and  modeling  to  show  the  impact  of  a  health 
monitoring  system  on  maintenance  activities.  That  said  a  review  of  relevant  maintenance 
optimization  and  planning  literature  is  still  insightful. 

Sherif  (1982)  provides  a  good  review  of  over  800  articles  in  which  he  defines 
eight  aspects  of  reliability  and  maintainability  that  methods  fall  under.  Additionally, 
Sherif  surmises  that  system  design  and  evaluation  must  consider  both  availability  and 
performance  criteria  to  meet  the  goal  of  maximizing  profit  and  availability  while 
minimizing  cost.  Most  research  however,  deals  with  failing  systems  and  neglects  the 
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prognostics  capability  of  ISHM  based  systems  driving  CBM  (Scarf,  1997)  (Huynh, 
Barros,  &  Berenguer,  2012)(Khac  Tuan  Huynh,  Barros,  &  Berenguer,  2012).  Chilcott 
and  Christer  (1991)  provide  an  examination  of  the  impact  of  CBM  on  manning  in  coal 
operations  assuming  perfect  diagnostic  capabilities  in  the  health  monitoring  system. 
Further,  Chilcott  and  Christer  find  that  even  small  downtime  savings  can  provide  large 
financial  benefits.  Mahulkar  et  al.  (2009)  present  a  study  of  manning  based  on  a  system 
of  systems  in  a  naval  environment  that  while  not  specifically  focused  on  condition  based 
maintenance,  does  include  the  principles  of  actions  based  on  component  or  system  states. 
Their  study  showed  intelligent  maintenance,  i.e.,  sensor  systems,  can  reduce  maintenance 
manning  requirements  and  increase  efficiency. 

Dekker  (1996)  points  out  that  while  a  number  of  mathematical  approaches  have 
been  proposed  to  approach  maintenance  optimization,  none  are  as  straight  forward  as  the 
economic  order  quantity  methods  used  in  supply  optimization.  He  goes  on  to  say  that 
most  maintenance  problems  require  software  and  models  to  predict  gains  in  maintenance 
methods  and  it  becomes  more  of  an  art  than  a  pure  science.  The  key  then  becomes 
adequately  and  accurately  measuring  the  model  outputs  to  gain  insight  into  the 
methodology. 

Maintenance  Metrics 

In  order  to  properly  measure  system  and  model  performance,  metrics  must  be 
established  and  defined  across  the  range  of  model  outputs.  First,  non-mission  capable  for 
maintenance  (NMCM)  is  an  aircraft  state  where  the  vehicle  is  operationally  incapable  due 
to  maintenance  requirements.  The  maintenance  man-hours  per  flying  hour  (MMH/FH) 
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metric  measures  the  effectiveness  of  maintenance  personnel  and  the  ease  with  which  a 
system  can  be  repaired.  Another  key  metric  in  determining  maintenance  efficacy  is  the 
mean  time  to  repair  (MTTR)  a  system,  that  is,  the  total  maintenance  elapsed  time  divided 
by  the  number  of  maintenance  actions  (Defense  Acquisition  University,  2012). 


MTTR  = 


Total  Maintenance  Time 
Number  of  Maintenance  Actions 


(12) 


Operational  availability  (Ao)  is  defined  as  the  system  availability  the  user  of  a 

system  realizes  (ReliaSoft,  2007).  Mathematically,  operational  availability  is  given  as: 

.  Uptime  MTBM 

An  =  - - -  =  -  ( 

Uptime  +  Downtime  MTBM  +  MMT  +  MLDT 

Where  MTBM  is  the  mean  time  between  maintenance,  MMT  is  the  mean  maintenance 

time,  and  MLDT  is  the  mean  logistics  delay  time.  For  the  purposes  of  this  research, 

MMT  and  MTTR  are  assumed  to  be  equivalent. 


Methodology 

Utilizing  the  military  aircraft  model  developed  in  (Vandawaker,  Jacques,  & 
Freels,  2015)  and  (Vandawaker,  Jacques,  Ryan,  Huscroft,  &  Freels,  2015)  by 
Vandawaker  et  ah,  this  research  explores  the  impact  of  health  monitoring  systems  on 
maintenance  manning.  They  compare  a  baseline  squadron  of  aircraft,  no  ISHM,  with  the 
same  squadron  with  ISHM  implemented,  examining  false  alarms  and  availability  impacts 
in  addition  to  cost.  Further,  they  explore  the  impact  of  ISHM  on  economic  order  quantity 
and  just-in-time  supply  methods.  The  just-in-time  supply  methodology  from 
(Vandawaker,  Jacques,  Ryan  et  ah,  2015)  is  utilized  for  stocking  components  in  this 
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research.  These  works  compared  ISHM  systems  to  baseline  systems  to  show  possible 
gains  through  the  use  of  health  monitoring  technology.  This  research  extends  the 
previous  modeling  approach  to  study  the  impact  of  maintenance  manning  levels  on  a 
system  equipped  with  health  monitoring  systems.  The  time-based  inspections  and 
preventive  maintenance  in  a  baseline  or  non-ISHM  equipped  aircraft  are  replaced  with 
computer  scans  of  sensor  data  which  are  then  used  to  predict  RUL  and  forecast 
maintenance. 

This  work  does  not  attempt  to  optimize  maintenance  manning,  but  to  show  the 
impact  of  an  ISHM  equipped  aircraft  squadron  on  manning  levels.  The  effect  of 
maintenance  personnel  levels  on  system  performance  is  only  observed  in  relation  to  the 
direct  maintenance  on  the  20  unique  components  modeled  on  each  of  12  aircraft  in  this 
work.  This  information  allows  planners  to  detennine  the  maintenance  requirements  and 
allocation  of  personnel,  or  fractions  thereof,  to  the  direct  support  of  the  researched  parts. 
For  each  of  the  twenty  components  it  is  assumed  that  the  time  between  successive 
failures  follows  an  exponential  distribution. 

The  ISHM  system  produces  prognostics  for  RUL  based  upon  its  calculations  from 
system  diagnostics.  There  are  two  components  in  the  RUL,  diagnosis  from  the  health 
monitoring  system  and  prediction  uncertainty.  In  this  research  component  diagnostics  is 
taken  as  perfect,  i.e.,  sensor  always  knows  exact  health.  With  many  health  monitoring 
techniques,  the  closer  a  component  is  to  failure,  the  better  uncertainty  can  be  quantified 
for  the  diagnosis  of  condition  (S.  Sankararaman  &  Goebel,  2013).  This  leads  to  the  line 
replaceable  unit  (LRU)  RUL  given  as: 


ill 


LRU  RUL-  \ognomrd\(  Diagnos  is  ^uncertainty) 


(14) 


Where  Diagnosis  is  equivalent  to  the  true  remaining  life  and  uncertainty  is  defined  in  Eq. 
(15). 

uncertainty  -  degredation  factor  +  In  {Part  RUL )  *  f uncertainty  factor  (15) 

where  Part  RUL  is  the  previous  RUL  prediction  for  that  part  and,  uncertainty  factor  is  a 
design  variable.  The  degradation  factor  places  an  additional  uncertainty  on  the  RUL 
prediction  given  as: 

degredation  factor  =  10  +  Part  ISHM  timer  /  20  (16) 

Where,  Part  ISHM  timer  is  the  accumulated  life  on  the  health  monitoring 
component  associated  with  a  specific  component.  The  leading  10  in  the  equation 
represents  a  fixed  initial  degradation  while  the  20  provides  a  representative  increase  in 
degradation  as  life,  or  wear,  incurs  on  each  part  respectively.  As  shown  above, 
degradation  factor  increases  in  direct  relation  to  the  accumulated  Part  ISHM  time.  The 
effect  of  the  degradation  factor  on  RUL  uncertainty  is  a  gradual  increase  in  the 
accumulated  uncertainty  for  a  particular  component.  When  a  component  is  replaced  or 
reaches  a  limit  in  the  number  of  false  alarms,  thus  triggering  ISHM  maintenance,  the 
ISHM  part  timer  is  reset  (Vandawaker,  Jacques,  &  Freels,  2015). 

In  this  research,  components  are  non-repairable,  that  is,  they  are  replaced  upon 
failure  or  an  indication  from  the  ISHM  system.  Upon  indication  of  impending  failure  by 
the  health  monitoring  system,  the  aircraft  is  brought  into  maintenance  for  inspection  to 
adjudicate  if  the  indication  is  real  or  a  false  alann.  The  times  to  complete  an  inspection 
for  all  components  are  triangularly  distributed  (20,  30,  45)  minutes.  If  the  inspection 
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confirms  the  ISHM  prognostics  the  component  is  then  replaced  along  with  others  deemed 
in  need  of  repair.  The  time  to  replace  each  line  replaceable  unit  (LRU)  replacements  are 
triangularly  distributed  (60,  90,  240)  minutes  if  only  1  maintainer  is  available  for  the 
action  and  triangularly  distributed  (40,  60,  160)  minutes  if  2  personnel  are  on  the  task. 
These  parameters  values  were  chosen  to  represent  a  range  of  repairs  and  inspections. 
Additionally,  if  multiple  components  require  replacement  or  inspection,  the  tasks  can  be 
completed  in  parallel,  if  sufficient  personnel  are  available. 

In  addition  to  components  being  replaced  upon  immediate  need,  maintenance  may 
also  occur  to  make  use  of  time  between  missions.  At  the  end  of  a  mission  the  ISHM 
system  scans  the  aircraft.  If  the  ISHM  system  indicates  maintenance  is  required  within  a 
predefined  maintenance  window  and  the  projected  time  to  complete  the  component 
replacement(s)  is  less  than  the  time  before  the  next  mission,  the  aircraft  is  brought  in  for 
maintenance.  Further,  if  an  aircraft  is  already  in  for  maintenance  and  the  ISHM  system 
indicates  other  components  are  not  in  need  of  immediate  replacement,  but  within  the 
maintenance  window,  those  components  can  be  replaced  if  there  is  adequate  stock. 

When  maintenance  actions  are  performed,  they  are  arranged  to  take  advantage  of 
available  personnel  to  complete  all  processes  in  the  quickest  manner.  Depending  on  the 
number  of  replacements  required  on  a  particular  aircraft,  these  tasks  may  be  performed  in 
parallel,  serially,  or  some  combination  of  the  two  utilizing  the  times  discussed  earlier  in 
this  section. 

Maintenance  personnel  are  assigned  to  one  of  two  12  hour  shift  manning  levels: 
day,  7AM  to  7PM;  and  night,  7PM  to  7AM.  Various  combinations  of  day  and  night  shift 
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manning  levels  are  explored  in  this  work,  ranging  from  1  day  and  0  night  shift  personnel 
to  20  maintainers  on  both  day  and  night  shift.  The  20  maintainer  case  provides  enough 
personnel  that  no  wait  time  for  inspection  or  repair  is  observed. 

The  modeling  assumptions  for  this  work  are  as  follows: 

•  Aircraft  missions  occur  around  the  clock  regardless  of  maintenance  manning; 

•  Aircraft  are  either  fully  mission  capable  (FMC)  or  non-mission  capable 
(NMC)  with  no  partial  mission  capability  (PMC). 

Results 

Reflected  in  this  section  are  the  results  of  manning  variations  across  model  runs. 
The  figures  herein  represent  the  two  shift  manning  levels  from  section  3.  The  upper  row 
of  the  x-axis  in  the  figures  contains  the  number  of  personnel  available  for  tasks  on  the 
day  shift  and  the  lower  row  includes  those  available  for  actions  on  the  night  shift.  The 
results  shown  below  are  the  mean  of  100  model  simulations  at  each  personnel 
combination.  The  analysis  for  statistical  significance  was  perfonned;  however, 
confidence  intervals  are  not  presented  in  the  figures  in  this  work.  The  reason  for 
confidence  intervals  not  being  presented  is  that  95%  intervals  shown  in  the  figures  are 
indistinguishable  from  the  mean  of  the  data,  which  are  shown. 

Model  Output 

Operational  availability  is  a  common  measure  of  system  performance.  The  A0’s 
observed  in  Figure  20  reflect  the  availability  of  the  squadron  of  12  aircraft  over  the 
duration  of  a  1 5  year  simulation.  The  addition  of  one  maintainer  to  the  night  shift  is 


shown  to  have  an  appreciable  impact  on  overall  system  perfonnance,  as  is  shown  further 
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in  the  remaining  figures.  For  the  condition  where  only  1  maintainer  is  available  on  the 
day  shift  and  none  on  the  night  shift,  Ao  is  slightly  above  0.90.  The  addition  of  one 
person  on  the  night  shift  to  provide  service  increases  the  Ao  to  over  0.96,  meaning  the 
aircraft  are  available  for  mission  tasking  an  additional  6%  of  the  time.  As  shown  in 
Figure  27,  that  6%  represents  an  additional  4.5  hours  of  flying  time  per  day  across  the  12 
aircraft  or  nearly  the  equivalent  of  adding  2  more  aircraft  to  the  squadron. 

Figure  20  also  shows  that  adding  1  additional  person  to  the  day  shift  for  a  total  of 
2  provides  a  benefit  worthy  of  consideration  by  management.  The  addition  of  other 
personnel  on  both  day  and  night  shifts  continues  to  have  positive  impacts  on  the 
operational  availability  but  with  smaller  increase  in  magnitude.  Beyond  the  inclusion  of  2 
personnel  on  the  night  shift,  Ao  gains  are  in  the  thousandths  and  when  factoring  in  the 
remaining  metrics  may  not  sway  management  decisions.  The  largest  gains  noted  in 
Figure  20  are  the  result  of  the  aircraft  not  having  to  wait  up  to  12  hours  over  the  night 
shift  for  maintenance  which  occurs  in  the  step  from  0  to  1  night  shift  maintainer.  Other 
gains  result  from  the  ability  to  complete  more  component  replacements  simultaneously  as 
more  personnel  are  available. 

While  the  A0  may  appear  high  for  many  aircraft,  it  should  be  noted  that  these 
results  only  include  the  impact  from  20  components  on  the  12  aircraft.  This  note  should 
also  be  considered  in  reviewing  the  remainder  of  the  data  and  figures  presented  in  this 
research. 
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Figure  26.  Operational  Availability 

Daily  flying  hours  for  the  12  aircraft  squadron,  shown  in  Figure  27,  reflect  that 
the  inclusion  of  just  1  maintainer  on  the  night  shift  has  the  effect  of  increasing  daily 
output  by  over  3  hours.  As  previously  mentioned,  this  is  the  equivalent  of  adding  another 
mission  to  the  daily  flying  schedule  for  the  cost  of  one  maintainer.  Further  gains  in  daily 
sortie  production  with  increasing  numbers  of  personnel,  both  day  and  night  shift,  are 
smaller  and  left  to  operations  and  maintenance  management  to  determine  the  efficacy  of 
including  the  extra  manpower. 
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Figure  27.  Daily  Flying  Hours 

Alluded  to  in  previous  discussion,  the  amount  of  time  annually  the  squadron  is 
NMCM  generally  decreases  as  the  number  of  personnel  increases  as  observed  in  Figure 
28.  The  addition  of  one  maintainer  to  the  night  shift  drops  the  mean  NMCM  from  10200 
hours  to  3400  for  the  1  maintainer  on  the  day  shift  case.  The  nearly  7000  hours  less  time 
down  for  maintenance  affords  increased  opportunities  for  aircraft  missions  as  well  as 
lower  workloads  for  the  individual  maintenance  personnel.  Greater  gains  can  also  be 
realized  with  the  inclusion  of  more  personnel  on  the  day  shift,  but  plateaus  as  the  NMCM 
time  becomes  mostly  hands-on  maintenance  with  little  to  no  awaiting  maintenance  time. 
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Figure  28.  Annual  NMCM  Hours 

Exploring  the  efficacy  of  maintenance  manning  from  a  different  vantage  than 
NMCM,  Figure  29  shows  the  average  time  an  aircraft  has  to  wait  prior  to  a  maintenance 
action  beginning.  In  other  words,  Figure  29  represents  the  responsiveness  of  the 
maintenance  system  to  aircraft  demands.  If  supply  wait  time  were  added  to  these  data,  the 
result  would  be  the  MLDT  from  Eq  (1).  The  shape  of  Figure  29  follows  that  of  Figure  28 
with  the  magnitude  reflecting  total  waiting  time  for  all  maintenance  actions  divided  by 
the  number  of  total  actions.  The  figure  clearly  shows  the  impact  of  the  addition  of  a  night 
shift  maintainer  on  the  wait  time  for  maintenance  to  begin,  bringing  the  average  wait  time 
from  over  5.5  hours  to  32  minutes.  Once  night  shift  personnel  are  increased  to  2  or  more, 
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the  average  wait  time  is  less  than  6  minutes  for  all  cases.  The  8  day  and  8  night  case  wait 
time  is  2  seconds  and  20  day,  20  night  is  0. 
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Figure  29.  Mean  Time  Waiting  for  Maintenance 

The  addition  of  repair  time  to  Figure  29  yields  the  maintenance  metric  of  MTTR 
found  in  Figure  30.  Again  it  is  noted  that  the  elimination  of  a  12  hour  period  where  no 
personnel  are  available  to  perfonn  maintenance  is  beneficial  to  aircraft  turn-around  time 
from  maintenance.  While  Figure  28,  Figure  29,  and  Figure  30  depict  similar  information, 
they  each  provide  insight  into  the  timeliness  of  the  maintenance  system  in  its  ability  to 
support  aircraft  operations. 
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As  in  Figure  29,  Figure  30  shows  an  area  of  little  variation  for  cases  where  2  or 
more  personnel  are  available  on  the  night  shift.  The  resulting  MTTRs  for  these  cases  are 
predominately  true  labor  hours  as  maintainers  are  available  to  perform  most  tasks  as  they 
arise. 
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Figure  30.  Mean  Time  to  Repair 

Figure  3 1  shows  the  utilization  rate  for  the  maintenance  personnel  assigned  to 
each  simulation.  The  first  column  shows  that  with  only  1  person  on  day  shift  and  none  on 
the  night  shift,  the  maintainer  is  busy  50%  of  the  available  hours.  While  that  may  not 
appear  to  be  overtaxing  on  the  individual,  that  person  must  perform  all  repairs  and 
inspections  as  required.  While  the  forward  looking  prognostics  of  the  ISHM  system  can 
allow  for  deconflicting  of  component  replacement,  failure  warnings  must  still  be 
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adjudicated  through  inspections  which  can  lead  to  backups  in  the  maintenance  process, 
noted  in  Figure  29. 
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Figure  31.  Personnel  Utilization 

The  doubling  of  day  shift  personnel  to  2  while  keeping  the  night  shift  at  0  results 
in  a  46%  drop  in  utilization  rate.  The  trend  in  Figure  3 1  shows  that  adding  personnel  on 
the  dayshift  steeply  drops  the  utilization  rate  while  including  additional  night  shift 
personnel  changes  the  magnitude  of  the  group  rates.  Examining  the  20  day  and  20  night 
case,  utilization  rate  drops  to  0.02,  leaving  those  personnel  98%  of  their  time  to  focus  on 
other  tasks. 
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The  usefulness  of  Figure  31  lies  in  being  able  to  manage  workload  for 
maintenance  personnel.  As  this  model  only  represent  20  components  there  are  likely 
other  tasks  maintainers  are  responsible  for  accomplishing.  If  it  is  shown  that,  in  the  case 
of  only  1  day  worker  and  0  night,  a  maintainer  will  need  to  focus  50%  of  their  shift,  or  6 
hours,  to  these  components  scheduling  other  work  becomes  more  difficult. 

The  amount  of  maintenance  required  for  a  flying  hour  is  an  indication  of 
efficiency  of  maintenance  processes  and  the  reparability  of  the  aircraft.  Figure  32  shows 
MMH/FH  remains  between  0.2  and  0.25.  While  MMH/FH  doesn’t  show  much  variation, 
the  entire  picture  is  not  clear  from  this  one  figure.  Total  maintenance  man  hours  are  lower 
for  the  cases  with  no  night  shift  personnel,  but  referring  to  Figure  27  it  is  also  noted  that 
flying  hours  are  lower  as  well.  The  driving  case  for  a  relatively  stable  MMH/FH  results 
from  the  repair  times  in  section  3,  which  do  not  vary  across  the  manning  scenarios.  It 
follows  then  that  as  the  cases  with  night  shift  personnel  have  more  flying  hours  the 
maintenance  hours  increase  accordingly,  keeping  the  MMH/FH  ratio  similar.  Within  the 
night  shift  bands,  MMH/FH  variation  is  noted  with  different  day  shift  personnel  numbers. 
When  observing  Figure  20  and  Figure  27  in  relation  to  Figure  32,  the  incremental  gains 
in  Ao  and  daily  hours  track  within  the  night  shift  bands. 
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Figure  32.  Maintenance  Man  Hours  per  Flying  Hour 

Turning  to  the  costs  associated  with  the  maintenance  of  the  20  components  of 
interest  in  this  research,  Figure  33  shows  the  annual  cost  of  direct  maintenance.  These 
figures  only  include  the  time  maintenance  personnel  are  actively  inspecting  components 
to  determine  if  failure  indications  are  true  or  are  in  the  process  of  replacing  items  on  the 
aircraft.  An  hourly  maintenance  cost  of  $50  is  used  for  all  personnel  (Department  of  the 
Air  Force,  February  1994).  The  annual  cost  of  maintenance  peaks  at  $155K  for  the  8  day, 
8  night  personnel  case  and  is  slightly  lower  for  the  20,  20  case,  due  to  the  efficiency  of 
having  more  personnel  to  complete  tasks.  The  lower  cost  of  the  1  day  and  0  night  case 
belies  the  fact  that  the  operational  output  is  markedly  lower,  as  shown  in  Figure  20  and 
Figure  27. 
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Figure  33.  Annual  Maintenance  Cost 

Shifting  focus  to  total  logistics  and  maintenance  cost  in  Figure  34,  it  is  clear  that 
the  direct  maintenance  costs  are  but  a  small  portion  of  the  overall  price  tag.  Except  in  the 
cases  where  no  night  shift  personnel  are  scheduled,  maintenance  costs  are  less  than  7%  of 
the  total  logistics  and  maintenance  cost.  The  percentage  rises  to  between  14%  and  21%  of 
the  total  cost  when  no  night  shift  personnel  are  available.  The  remainder  of  this  cost  is  the 
purchase  and  storage  of  spare  parts  for  aircraft  operation. 
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Figure  34.  Annual  Logistics  and  Maintenance  Cost 


Case  Study 

Let  us  explore  the  data  in  section  4.1  from  a  maintenance  management 
perspective.  If  a  decision  had  to  be  made  where  only  2  personnel  were  available  for 
maintenance  on  the  20  components  studied  which  shift  combination  is  preferred:  2  day 
and  0  night;  or  1  day  and  1  night.  We  will  assume  that  management  wants  the  most 
performance,  Ao  and  daily  flying  hours,  for  the  least  cost  with  other  factors  considered 
for  comparison. 

Daily  flying  hours  for  the  (2,  0)  case  are  29.3  versus  32.7  for  the  (1,  1)  case,  with 
operational  availabilities  of  0.93  and  0.96  respectively.  With  only  these  two  items 
considered,  management  would  choose  the  (1,  1)  case  as  better  performance  is  achieved. 
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When  cost  is  considered,  annual  maintenance  costs  are  $10000  larger  for  the  (1,1)  case 
or  $150000  over  the  15  year  simulation  time  with  a  $230000  total  logistics  and 
maintenance  cost  difference.  The  question  for  management  then  becomes  is  this  cost 
increase  justified  by  the  higher  system  performance? 

The  likely  answer  to  the  previous  question  is  yes  as  the  system  performance  gain 
of  the  (1,1)  case  over  the  (2,  0)  case  is  equivalent  to  an  additional  aircraft  worth  of  flying 
time  each  day.  Thus  if  the  cost  of  an  aircraft  is  higher  than  $230000  with  a  planned  life  of 
at  least  15  years  the  higher  support  costs  can  be  justified.  Similarly,  an  aircraft  could  be 
removed  from  the  (1,1)  case  and  still  achieve  performance  equivalent  to  the  (2,  0)  case. 

When  additional  factors,  shown  in  the  remaining  figures,  are  considered,  the  case 
for  choosing  a  (1,  1)  maintenance  manning  scheme  becomes  greater:  4000  fewer  NMCM 
hours  per  year;  190  minutes  less  wait  time  for  maintenance;  3  hours  less  MTTR;  and  a 
slightly  smaller  MMH/FH.  The  personnel  utilization  rate  of  the  (1,1)  case  is  0.29  versus 
0.27  for  the  (2,  0)  case,  which  could  be  a  positive  or  negative  depending  on  other 
requirements  placed  on  the  personnel. 

In  summary,  a  management  decision  based  on  2  maintainers  would  likely  yield  a 
1  day  and  1  night  shift  scheme  based  on  the  evidence  presented  above.  Further  utility  in 
this  research  is  offered  in  determining  maintenance  workload  and  the  impacts  of  manning 
decisions  on  overall  aircraft  system  performance.  Additionally,  the  model  parameters  for 
health  monitoring  capabilities  can  be  changed  to  determine  the  downstream  effect  of 
system  design  on  the  logistics  and  maintenance  operations.  This  information  can  then  be 
used  to  provide  trade-off  analyses  to  program  management. 
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Conclusion 


The  work  presented  above  provides  a  tool  for  managers  and  planners  for 
utilization  in  system  design  and  support  planning.  Information  captured  in  the  figures 
details  cost  and  perfonnance  data  that  show  efficiency  in  maintenance  processes  at  set 
manning  levels.  The  ability  to  define  a  trade-space  for  system  parameters  early  in  the 
development  of  an  aircraft  or  any  system  affords  the  opportunity  to  explore  future  cost 
savings  and  weigh  performance  trade-offs. 

Future  work  in  this  model  environment  should  include  additional  study  of 
maintenance  windowing  for  when  aircraft  is  already  being  brought  in  for  other  work. 
Also,  the  utilization  of  remaining  useful  life  prognostics  could  be  enhanced  for  more 
detailed  event  scheduling  for  further  gains  in  downtime  efficiency  and  aircraft  utilization. 
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VII.  Component  Replacement  Windowing 


Chapter  Overview 

This  chapter  examines  the  impact  of  placing  a  maintenance  opportunity  timeframe 
(window),  counting  back  from  the  projected  component  failure  time,  in  the  maintenance 
logic.  This  window  affords  the  opportunity  for  components  to  be  replaced  if  they  fall 
within  the  specified  time  frame,  and  the  aircraft  is  already  in  maintenance  for  another 
replacement  or  inspection.  The  data  presented  herein  show  there  are  potential  benefits,  in 
both  cost  and  availability  to  establishing  this  type  of  maintenance  window. 

Model  Description 

The  parameters  used  in  this  chapter  are  the  same  as  those  of  chapter  6  with  the 
addition  of  the  maintenance  opportunity  introduced  above.  While  getting  all  useful  life 
out  of  a  component  is  desirable,  in  some  instances  it  may  make  sense  to  replace  one  or 
more  components  earlier  than  planned.  The  reasons  can  be  for  both  availability  and  cost 
savings  in  the  long  term  as  maintaining  several  components  at  once  is  generally  more 
efficient  than  replacing  them  individually.  This  grouping  of  maintenance  actions  saves  on 
set-up  time  for  repairs  as  it  only  has  to  be  done  once  versus  many  times  for  individual 
components.  The  results  below  illustrate  the  benefits  and  drawbacks  of  a  maintenance 
window  and  its  effect  on  cost  and  availability. 
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Results 


This  work  examines  results  across  a  range  of  maintenance  manning  scenarios 
initially,  as  in  chapter  6,  to  show  the  effect  on  operational  availability  and  daily  flying 
hours.  The  number  of  maintenance  personnel  on  the  day  and  night  shifts,  and  the  length 
(lead  time)  of  the  maintenance  interval  will  be  varied  to  examine  this  effect.  The 
remainder  of  the  figures  show  only  one  manning  scenario  to  focus  on  the  impact  of 
various  levels  of  maintenance  windowing. 

Figure  1  depicts  the  operational  availability  of  the  twelve  aircraft  squadron  across 
a  range  of  manning  scenarios  and  maintenance  windows.  It  is  observed  in  the  figure  that 
as  the  number  of  personnel  available  increases  Ao  does  likewise.  Within  each  manning 
band  six  maintenance  windows  are  also  graphed  for  comparison.  The  effect  of  the 
maintenance  opportunity  window  on  Ao  shows  an  increase  until  40  hours  prior  to 
projected  failure  and  then  a  slight  reduction  at  50  hours.  As  in  chapter  6,  the  increasing 
A0  trend  across  the  manning  bands  is  as  a  result  of  personnel  available  to  perform  repairs 
as  required  instead  of  aircraft  sitting  and  waiting  for  inspections  or  component 
replacements.  Within  each  manning  band  the  operational  availability  increases  as  the 
window  increases,  until  it  reaches  50  hours.  This  A0rise  results  from  seizing  the 
opportunity  to  perform  maintenance  when  the  aircraft  is  already  down  for  another  action, 
thus  saving  on  inspection  and  set-up  time  to  perform  replacements  individually.  The 
leveling  off  of  A0  between  30  and  50  hours  results  from  more  replacements  being 
conducted,  especially  for  components  that  have  low  MTBFs,  which  require  more 
downtime  than  utilizing  a  smaller  window. 
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Figure  35.  Maintenance  Windowing  Operational  Availability 

Mean  daily  squadron  flying  hours,  shown  in  Figure  36,  follow  the  general  trends 
seen  with  operational  availability.  As  in  the  manpower  study  in  chapter  6,  daily  hours 
increase  as  more  personnel  are  available  to  maintain  the  aircraft.  Within  each  manning 
band,  the  effect  of  maintenance  opportunity  windowing  is  noted  to  increase  as  the 
window  hours  increase.  The  gains  in  daily  hours  associated  with  the  different  windows 
result  from  the  opportunity  taken  to  incur  fewer  times  down  to  make  component 
replacements  in  groups  versus  individually.  The  reason  daily  flying  hours  do  not  drop  off 
at  the  same  time  operational  availability  does  lies  chiefly  in  the  model  design.  When  an 
aircraft  returns  to  mission  availability  following  being  in  maintenance  it  incurs  a  hold 
prior  to  its  next  mission.  While  this  hold  time  is  uptime  in  terms  of  A0,  flying  does  not 
occur  and  thus  daily  hours  do  not  accrue.  Shown  later  in  Table  6,  false  alarms  drive  the 
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number  of  times  aircraft  enter  maintenance  at  lower  maintenance  windows.  Once  false 


alarms  are  cleared,  the  aircraft  is  released  back  to  the  hold  status  until  its  next  mission  is 
scheduled.  Therefore,  the  more  false  alarms  that  occur,  the  more  times  an  aircraft  enters 
the  mission  hold  sequence  thereby  reducing  the  number  of  hours  the  aircraft  fly. 

38  r 

Window  (hrs) 


1  2  2  4  4  8  20 

1  2  4  8  20 


Maintenance  Personnel  Available  (Day /Night) 

Figure  36.  Mean  Daily  Flying  Hours 

From  this  point  forward  the  figures  and  discussions  examine  the  maintenance 
manning  case  with  4  personnel  on  the  day  shift  and  2  personnel  on  the  night  shift.  This 
case  is  chosen  since  overall  gains  in  hours,  availability,  and  costs  are  relatively  low  for 
the  scenarios  with  additional  personnel  available. 

One  of  the  goals  of  grouping  maintenance  activities  is  to  reduce  the  total  number 
of  hours  the  aircraft  are  down  and  unavailable  for  operations.  Figure  37  shows  the  mean 
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annual  NMCM  time  for  each  maintenance  window.  It  is  observed  that  as  the  window  size 


increases,  that  is,  a  greater  opportunity  to  replace  a  component  before  its  predicted 
failure,  NMCM  hours  decline.  The  decrease  is  attributed  to  two  items.  First,  grouping  of 
component  replacements  into  a  single  maintenance  action  in  lieu  of  several  maintenance 
actions  with  only  one  component  replaced.  Second,  as  components  are  replaced  more 
frequently  as  the  window  widens  the  ISHM  system  is  reset  more  frequently  which  leads 
to  the  added  benefit  of  fewer  false  alarms  which  require  inspections  to  clear.  From 
chapters  4  and  6  it  is  noted  that  as  the  sensors  accumulate  flight  hours,  they  also  degrade 
which  leads  to  an  increase  in  the  number  of  false  alarms.  In  this  scenario,  false  alarms 
decrease  from  an  average  of  248  annually  to  22  when  the  maintenance  window  widens 
from  1  to  50  hours.  The  yearly  NMCM  hours  for  a  50  hour  window  are  52%  lower  than 
at  a  1  hour  maintenance  window  and  50%  lower  than  a  10  hour  window.  Stated  another 
way,  across  the  15  year  simulation  the  50  hour  maintenance  window  provides  over  36000 
hours  more  availability  as  the  aircraft  are  no  longer  NMCM. 
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Figure  37.  Mean  Annual  Non-Mission  Capable-Maintenance  Hours 

The  reduction  in  total  NMCM  hours  with  an  increasing  maintenance  window, 
made  possible  by  a  grouping  of  maintenance  tasks,  has  a  somewhat  inverse  trend  in  mean 
downtime.  As  Figure  38  shows,  mean  downtime  increases  as  the  maintenance  window 
widens.  At  a  50  hour  window,  the  MDT  is  more  than  double  that  at  either  1  or  10  hour 
windows.  Grouping  of  component  replacements  into  larger  aggregate  tasks  versus  many 
individual  ones  increases  the  overall  time  required  each  time  down,  but  as  Figure  37 
shows  the  net  effect  is  lower  overall  system  downtime.  While  this  stands  in  contrast  to 
DoD  predictions  of  “significantly  reduced”  MDTs  by  performing  CBM  versus  time- 
based  preventive  maintenance  it  does  not  necessarily  contradict  the  assertion  (Under 
Secretary  of  Defense  (AT&L),  May  2008).  Grouping  takes  advantage  of  the  diagnostic 
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and  prognostic  capabilities  provided  by  ISHM  to  apply  CBM  in  a  more  optimal  manner 
to  reduce  total  aircraft  downtime  in  lieu  of  many  shorter  downtimes  enabled  by  the  same 
ISHM  capabilities.  The  net  effect  of  the  grouping  of  maintenance  activities  is  to  increase 
the  mean  downtime  while  simultaneously  reducing  total  downtime.  There  is  an  additional 
increase  in  MDT  as  a  result  of  a  growth  in  NMCS  time  as  the  maintenance  window 
becomes  larger.  The  NMCS  increase  results  from  the  supply  ordering  system,  in 
particular  the  supply  lead  time,  not  keeping  up  with  the  opportunistic  maintenance 
occurring  during  the  grouping  windows. 
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Figure  38.  Mean  Downtime 


As  the  maintenance  opportunity  window  widens,  a  counterpoint  is  developed  in 
an  increase  in  the  unused  useful  life  for  the  components.  Figure  39  shows  the  average 


134 


useful  life  lost  across  all  components.  The  values  are  determined  by  taking  the  difference 
between  the  accumulated  part  life  at  replacement  and  the  actual  (not  ISHM  RUL 
projection)  failure  point  assigned  to  new  parts.  Major  factors  that  contribute  to  life  lost 
include  the  RUL  prediction  error,  maintenance  window,  safety  factor,  projected  mission 
duration,  stock  levels,  and  manning  levels  among  others.  Safety  factor  is  fixed  at  10 
hours  for  all  components  and  mission  durations  are  LOGNORM  (4,  3)  hours.  These  two 
components  along  with  error  in  RUL  prognostics  account  for  the  majority  of  part  life  lost. 
The  accumulation  of  the  items  above  leads  to  the  information  presented  in  Figure  39, 
which  shows  that  useful  life  lost  increases  142%  from  a  1  hour  window  to  a  50  hour 
window.  While  it  may  be  desirable  from  a  utility  standpoint  to  get  all  the  useful  life 
available  from  a  component,  it  may  not  be  the  best  financial  decision. 
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Figure  39.  Average  Life  Lost  Per  Component 
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Examining  Figure  40  shows  that  as  the  maintenance  window  increases,  supply 
costs  go  up  as  well.  Ideally,  costs  should  be  kept  as  low  as  possible,  but  a  balance  must 
be  struck  between  cost,  availability,  and  system  performance.  Following  the  trend  from 
Figure  39,  annual  supply  cost  rises  as  maintenance  window  increases  due  to  a  larger 
demand  for  new  components  to  replace  those  being  removed  from  service  under  the 
maintenance  opportunity  window.  Obviously,  one  would  expect  a  correlation,  and 
causation,  between  RUF  and  supply  cost.  If  parts  are  being  replaced  more  often,  more 
parts  will  need  to  be  purchased  over  the  system  lifetime.  Further  exploration  of  cost 
should  be  considered  to  develop  a  trade-space  with  the  parameters  discussed  above. 
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Figure  40.  Annual  Supply  Cost 
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Figure  41  contains  the  average  annual  maintenance  costs  for  the  maintenance 
opportunity  windows.  It  is  noted  that  the  cost  of  maintenance  is  considerably  lower  than 
that  of  spares.  Maintenance  costs  captured  are  only  those  directly  related  to  work  on  the 
aircraft  performing  inspections,  replacements,  etc.  Idle  costs  and  those  associated  with 
training  and  other  duties  are  not  captured  in  this  research.  Figure  41  shows  that 
maintenance  costs  decrease  as  the  maintenance  window  increases.  This  cost  decrease 
results  from  the  reduced  set-up  time  for  grouped  maintenance  actions  versus  individual 
ones. 
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Figure  41.  Annual  Maintenance  Cost 
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Combining  the  data  from  Figure  40  and  Figure  41  yields  the  total  annual  cost 
found  in  Figure  42.  As  the  maintenance  costs  are  much  lower  than  supply  costs,  total  cost 
is  dominated  by  the  cost  to  procure  and  store  parts.  Over  the  15  year  simulation  these 
annual  costs  result  in  a  $7.5M  difference  between  the  1  hour  maintenance  window  and 
the  50  hour  window,  with  the  50  hour  window  total  cost  being  $42. 3M.  As  previously 
mentioned  these  costs  need  to  be  weighed  along  with  availability  data  and  compared  to 
program  objectives  to  determine  a  proper  maintenance  window. 
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Figure  42.  Total  Annual  Cost 


An  examination  of  the  trade-space  between  Ao,  total  cost,  and  component  life  lost 
shows  the  decisions  program  managers  face  when  selecting  a  set  of  maintenance 
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parameters.  Figure  43  compares  annual  costs  to  average  component  lost  life  for  the  6 
maintenance  windows  utilized.  Restating  from  prior  discussion,  it  is  observed  that  as  the 
window  increases,  component  life  lost  and  total  cost  both  rise.  This  figure  by  itself  would 
likely  drive  a  decision  to  select  the  smallest  window,  but  this  does  not  provide  a  complete 
picture  of  what  is  happening  system  wide. 
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Figure  43.  Annual  Cost  versus  Component  Life  Lost 


Figure  44  provides  another  component  in  the  trade-space  analysis  by  comparing 
operational  availability  to  component  life  lost.  While  Figure  43  showed  the  lowest  cost  at 
the  smallest  maintenance  window,  Figure  44  shows  it  also  produces  the  lowest  Ao.  This 
introduces  a  decision  point  for  management  to  determine  whether  the  0.95  Ao  of  the  1 
hour  window  is  acceptable  or  if  0.97  is  more  desirable.  It  is  noted  that  these  operational 
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availabilities  are  only  for  the  impact  of  the  20  components  studied  and  not  for  all  aircraft 
systems  as  a  whole. 
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Figure  44.  Operational  Availability  versus  Component  Life  Lost 

A  final  comparison,  found  in  Figure  45,  shows  operational  availability  contrasted 
with  total  cost.  Utilizing  this  figure,  decisions  can  be  made  based  upon  the  cost  to  achieve 
increases  in  Ao  for  the  various  maintenance  opportunity  windows.  It  is  observed  that  an 
increase  from  0.95  at  the  1  hour  window  to  0.97  at  the  30  hour  window  will  cost  an 
additional  $295K  annually  or  $4.4  M  over  the  15  year  simulation.  It  is  left  to 
management  and  program  objectives  to  detennine  if  this  trade-off  is  beneficial  to  overall 
program  goals. 
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Figure  45.  Operational  Availability  versus  Annual  Cost 


Examining  the  material  reliability  metric  of  MTBF  for  the  aircraft  is  not  possible 
for  this  ISFIM  research  as  no  component  failures  are  recorded.  The  design  of  the  ISFIM 
monitoring  logic  combined  with  adjudication  of  false  alarms  and  recalibration  of  true 
component  status  capture  pending  failures  before  they  occur.  Additionally,  a  10  flight 
hour  safety  factor  is  included  in  RUL  prognostics  thus  adding  extra  margin  to  component 
failure.  While  MTBF  may  not  be  measureable,  mean  time  between  maintenance 
(MTBM)  and  mean  time  between  repair/replacement  (MTBR)  for  the  aircraft  can  be 
calculated. 

Aircraft  MTBR  is  determined  by  taking  accumulated  flight  hours  and  dividing  by 
the  number  of  replacements  or  groups  of  replacements  to  find  average  time  the  aircraft  as 
a  whole  are  available  between  fixes.  Figure  46  shows  MTBR  times  for  the  maintenance 
windows  utilized  in  this  work.  In  this  scenario  a  20  hour  window  has  the  highest  MTBR 
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by  25  minutes  over  the  30  hour  window.  Referring  back  to  Figure  36  it  is  shown  that  as 
the  maintenance  window  increases  daily  flying  hours  do  as  well.  That  information, 
coupled  with  Figure  46,  must  mean  that  more  replacements  are  occurring  since  the 
MTBR  is  dropping.  This  is  a  logical  inference  since  as  more  hours  are  flown,  components 
will  experience  a  corresponding  increase  in  use  thus  requiring  replacement  more  often. 


Maintenance  Window  (hrs) 

Figure  46.  Aircraft  Mean  Time  Between  Replacement 


Aircraft  MTBM  is  the  average  of  the  accumulated  flight  time  from  when  the 
aircraft  leaves  maintenance  until  it  returns  for  any  reason;  inspection,  false  alarm,  repair, 
etc.  Figure  47  shows  the  aircraft  MTBM  which  increases  as  the  maintenance  window 
widens.  The  trend  tracks  with  the  inverse  of  NMCM  in  Figure  37.  As  the  NMCM 
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decreases  with  increasing  window  size  due  primarily  to  the  grouping  of  maintenance 
activities,  so  does  the  number  of  times  the  aircraft  enter  maintenance  control.  This  results 
in  longer  durations  between  inspections  and  replacements.  The  added  benefit  of  the  more 
frequent  component  replacement  with  an  increasing  maintenance  window  is  the  reduction 
in  false  alarms.  False  alarms,  at  the  1  hour  window,  dominate  the  cause  of  entry  into 
maintenance,  which  drives  the  MTBM  down. 
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Figure  47.  Aircraft  Mean  Time  Between  Maintenance  Action 

False  alarms  drive  the  number  of  times  aircraft  enter  maintenance  in  the  smaller 
maintenance  windows,  with  the  percentage  decreasing  as  the  maintenance  window 
grows.  Table  6  shows  the  relationship  between  maintenance  window,  number  of  times 
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entering  maintenance,  false  alanns,  and  number  of  times  components  are  replaced  The 
number  of  replacements  counts  either  individual  component  replacements  or  a  group  of 
component  replacements  as  one  action.  The  number  of  replacements  varies  mainly  as  a 
result  of  the  maintenance  window  and  the  impact  of  the  window  on  daily  flying  hours 
shown  in  Figure  36.  False  alanns  decrease  because  as  components  are  replaced  more 
frequently,  and  further  from  projected  failure,  the  ISHM  sensor  and  prognostic 
degradation  does  not  impact  the  RUL  projection  as  greatly.  The  difference  in  false 
alarms,  along  with  their  downtime  impact  on  aircraft  flying  hours,  and  the  effect  of 
maintenance  grouping,  accounts  for  the  difference  between  MTBR  and  MTBM. 


Table  6.  Maintenance  Actions 


Maintenance 
Window  (hrs) 

Times  Aircraft 

Enter 

Maintenance 

False  Alarm  as 

Cause 

Number  of 
Replacement 
Actions 

1 

42303 

82.6% 

7378 

10 

39739 

83.0% 

6757 

20 

26257 

73.0% 

7077 

30 

17713 

57.2% 

7576 

40 

13511 

40.8% 

7999 

50 

11697 

28.7% 

8344 

As  shown  in  Table  6  the  percentage  of  false  alanns  as  the  sole  cause  of  an  aircraft 
entering  maintenance  is  high.  There  are  two  means  of  entry  for  an  aircraft  into 
maintenance.  The  first  entry  case  results  from  an  ISHM  system  indication  that  a  potential 
fault  exists  in  one  or  more  of  the  aircraft  components  that  will  violate  the  safety  limit  or 
fail  within  the  projected  flight  time.  If  an  aircraft  is  found  to  have  only  false  alarms  after 
inspection,  the  maintenance  opportunity  window  is  not  checked  and  the  aircraft  is 
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released  to  perfonn  missions.  However,  if  one  component  triggers  a  replacement,  all 
components  on  the  aircraft  within  the  maintenance  window  then  are  eligible  for 
replacement.  The  second  entry  case  for  an  aircraft  into  maintenance  occurs  post  mission 
during  the  hold  before  the  next  scheduled  flight.  A  maintenance  window  check  is 
performed  when  an  aircraft  reaches  the  post  mission  hold.  If  any  projected  repairs  can  be 
completed  prior  to  the  next  mission,  the  aircraft  is  routed  to  maintenance  for  component 
replacements.  This  between  mission  check  provides  the  chance  to  capture  maintenance 
actions  during  nonnal  standby  time  for  the  aircraft  prior  to  the  next  mission.  As  the 
maintenance  window  increases,  this  hold  check  for  potential  maintenance  actions  is 
increasingly  the  cause  of  aircraft  entering  maintenance. 

Aircraft  entering  maintenance  from  the  between  mission  hold  check  have  cleared 
the  post  flight  ISHM  scan  without  any  indications  of  potential  failures  within  the  safety 
window.  Since  the  aircraft  from  the  hold  haven’t  been  sent  to  maintenance  as  a  result  of 
immediate  need  for  repair,  only  sent  for  opportune  maintenance  within  the  maintenance 
window,  they  forgo  the  inspection  and  false  alann  check  and  proceed  directly  for 
replacement.  This  leads  to  components  being  replaced  further  from  failure  and  prior  to 
triggering  most  false  alarms.  There  are,  however,  multiple  RUL  checks  per  mission  so 
this  hold  maintenance  under  the  maintenance  window  does  not  eliminate  all  false  alarms. 
Additionally,  as  the  uncertainty  grows  with  sensor  life  (see  Eq.  (5)),  the  potential  for  false 
alarms  increases  as  well. 

A  parametric  analysis  was  performed  with  static  prognostic  uncertainty  levels, 
without  degradation,  for  the  model  confirming  the  increase  in  false  alarms  as  uncertainty 
rises,  regardless  of  maintenance  window  size.  Further,  as  the  uncertainty  level  and  thus 
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the  standard  deviation  of  the  lognormal  distribution  used  for  RUL  increases,  the 
probability  that  the  RUL  prognostic  will  predict  less  than  the  true  remaining  life 
increases.  This  shift  is  a  function  of  the  lognormal  distribution  and  the  impact  its  standard 
deviation  has  on  skewing  the  shape  of  the  distribution.  As  the  standard  deviation 
increases,  the  mode,  or  peak,  of  the  distribution  shifts  to  the  left  and  decreases  in 
magnitude  while  the  right  tail  decrease  more  slowly,  increasing  the  probability  that  the 
RUL  prediction  could  be  much  greater  than  actual  remaining  life.  Further,  analysis  of 
prognostic  uncertainty  shows  that  if  the  lognonnal  function  is  utilized  for  RUL  prediction 
uncertainty  must  be  held  to  a  low  level  in  the  system.  If  uncertainty  becomes  too  large, 
the  system  risks  cost  effectiveness  resulting  from  too  many  inspections  to  clear  false 
alarms  and  general  mistrust  by  users. 

Shown  in  Table  7  below  are  the  results  of  the  parametric  analysis  of  prognostic 
uncertainty.  The  left  two  columns,  uncertainty  and  maintenance  window,  are  the 
parameters  of  change  in  the  simulation.  The  remaining  columns  are  model  outputs  related 
to  aircraft  entering  maintenance  and  the  associated  reason  for  entering.  It  is  noted  that  as 
the  uncertainty  increases,  the  percentage  of  time  the  only  reason  an  aircraft  enters 
maintenance  is  for  a  false  alarm  increases  considerably.  Above  an  uncertainty  level  of  10, 
the  percentage  of  false  alarms  is  likely  unacceptable  for  a  fielded  system.  Shifting  focus 
to  the  maintenance  window  factor;  it  is  shown  in  the  between  mission  replacements 
column  that  as  the  maintenance  window  increases,  the  number  of  times  the  aircraft  enter 
maintenance  from  the  hold  station  increases  as  well.  This  results  from  the  widening 
window  capturing  more  components,  further  from  actual  failure,  to  replace  during  this 
opportunity. 
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Table  7.  False  Alarm  Parametric  Analysis 


Static 

Uncertainty 

Maintenance 

Window 

(hrs) 

False  Alarm 
Only  Cause 
for  Entry 

Number  of 
Times  Enter 
Maintenance 

Number  of 
Maintenance 
Actions 

Between 

Mission 

Replacements 

False  Alarm 
Entry 
Percentage 

1 

1 

0 

8083 

8083 

0 

0.00% 

1 

10 

0 

7132 

7132 

0 

0.00% 

1 

20 

0 

6436 

6436 

4578 

0.00% 

1 

30 

0 

6517 

6517 

5744 

0.00% 

1 

40 

0 

6600 

6600 

6056 

0.00% 

1 

50 

0 

6771 

6771 

6280 

0.00% 

5 

1 

10 

8351 

8342 

0 

0.11% 

5 

10 

9 

7635 

7626 

0 

0.12% 

5 

20 

5 

6841 

6836 

3555 

0.07% 

5 

30 

2 

6659 

6657 

5481 

0.03% 

5 

40 

1 

6798 

6797 

6091 

0.01% 

5 

50 

0 

6968 

6968 

6382 

0.00% 

10 

1 

1767 

10125 

8358 

0 

17.45% 

10 

10 

1706 

9596 

7891 

0 

17.77% 

10 

20 

1025 

8278 

7253 

3102 

12.39% 

10 

30 

403 

7503 

7100 

5315 

5.37% 

10 

40 

144 

7321 

7177 

6213 

1.97% 

10 

50 

73 

7348 

7275 

6537 

1.00% 

20 

1 

15813 

23777 

7964 

0 

66.51% 

20 

10 

15138 

22694 

7556 

0 

66.70% 

20 

20 

8820 

16075 

7254 

3338 

54.87% 

20 

30 

4202 

11614 

7412 

5418 

36.18% 

20 

40 

1968 

9537 

7569 

6366 

20.63% 

20 

50 

1073 

8815 

7742 

6814 

12.18% 

30 

1 

33200 

40783 

7582 

0 

81.41% 

30 

10 

31592 

38670 

7078 

0 

81.70% 

30 

20 

18167 

25301 

7133 

3582 

71.81% 

30 

30 

9377 

16881 

7504 

5603 

55.55% 

30 

40 

4955 

12733 

7779 

6533 

38.91% 

30 

50 

3082 

11091 

8009 

6994 

27.79% 

40 

1 

47587 

54806 

7219 

0 

86.83% 

40 

10 

44897 

51599 

6702 

0 

87.01% 

40 

20 

26385 

33407 

7021 

3653 

78.98% 

40 

30 

14280 

21768 

7488 

5645 

65.60% 

40 

40 

8109 

15949 

7840 

6585 

50.84% 

40 

50 

5458 

13594 

8136 

7064 

40.15% 
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Statistical  Significance 

An  analysis  for  statistical  significance  was  perfonned;  however,  confidence 
intervals  are  not  presented  in  the  figures  in  this  work.  The  reason  for  confidence  intervals 
not  being  presented  is  that  95%  intervals  shown  in  the  figures  are  not  distinguishable 
from  the  mean  of  the  data,  which  are  shown.  This  holds  true  for  the  entirety  of  this 
research.  For  the  data  presented  in  chapter  7,  with  the  exception  of  the  10  and  40  hour 
maintenance  windows  MTBR  data  in  Figure  46,  the  p-values  for  t-test  comparisons 
within  each  figure  are  less  than  0.0001,  thus  statistically  significant.  The  data  for  10  and 
40  hour  windows  can  be  compared  in  other  figures,  with  statistical  significance,  to 
determine  which  one  to  choose.  Additionally,  as  each  of  the  100  replications  run  per 
scenario  has  12  aircraft  and  simulates  15  years,  an  immense  amount  of  data  points  are 
generated  yielding  the  small  confidence  intervals. 

Summary 

This  chapter  illustrates  the  potential  benefit  of  including  a  maintenance 
opportunity  window  to  allow  grouping  of  maintenance  tasks.  It  is  shown  that  grouping 
can  improve  operational  availability  through  a  reduction  in  total  downtime.  However,  this 
performance  improvement  comes  at  the  expense  of  increased  spares  costs  even  while 
maintenance  costs  decline.  The  maintenance  window  pennits  another  factor  to  be 
considered  in  perfonning  a  trade-space  analysis  to  determine  requirements  for  ISHM 
capabilities  and  maintenance  and  logistics  processes.  Finally,  data  show  that  uncertainty 
needs  to  be  kept  low  if  a  lognonnal  distribution  is  used  for  the  remaining  useful  life 
prognostic. 
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VIII.  Conclusions  and  Recommendations 


Chapter  Overview 

This  chapter  compiles  conclusions  from  the  preceding  chapters  and  identifies  the 
significance  of  the  research  presented.  Recommendations  for  action  and  proposed  future 
research  efforts  are  also  discussed. 

Conclusions  of  Research 

The  main  contribution  of  this  effort  is  a  simulation  tool  to  compare  sensing  and 
maintenance  options  and  examine  their  impact  on  desired  performance  factors.  The 
ability  to  input  ISHM  system  and  aircraft  characteristics  and  investigate  alternative 
approaches  to  monitoring,  maintenance  processes,  and  spares  management  makes  this 
tool  useful  in  program  decisions  on  whether  or  not  to  implement  monitoring  techniques. 
Weighing  the  impacts  of  simulation  outputs  and  management  philosophies  with  cost  and 
performance  objectives  is  left  to  program  managers  to  detennine  the  appropriate  level  of 
service  required.  The  ability  to  define  a  trade-space  for  system  parameters  early  in  the 
development  of  an  aircraft  or  any  system  affords  the  opportunity  to  explore  future  cost 
savings  and  weigh  performance  trade-offs  against  defined  system  architectures  and 
component  system  requirements. 

Conclusions  from  this  research  and  answers  to  the  research  questions  from 
chapter  1  are  as  follows:  Question  #1  asked  what  the  key  cost  and  effectiveness  drivers 
for  ISHM  enabled  condition  based  maintenance  and  logistics  processes.  The  research 
scenarios  in  chapters  4  through  7  show  that  prognostic  accuracy  is  a  driver  of  cost  to 
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operate  with  an  ISHM  system  in  that  it  drives  false  alarms  and  thus  unnecessary 
inspections  and  labor  costs.  In  order  to  counter  a  set  or  known  prognostic  accuracy 
setting  a  tolerance  for  the  number  of  false  alarms  allowed  prior  to  ISHM  system 
maintenance  becomes  a  trade-off  between  cost  and  system  availability.  Further, 
maintenance  personnel  allocation  and  maintenance  opportunity  windows  impact  cost  and 
availability  and  must  be  weighed  when  detennining  system  priorities.  Additionally,  with 
ISHM  prognostics,  supply  ordering  gains  efficiency  over  baseline  processes  and  with 
careful  contract  negotiations  a  just-in-time  logistics  chain  may  be  a  viable  alternative. 
Finally,  cost  of  development,  acquisition,  and  implementation  of  the  ISHM/CBM  system 
is  a  major  driver  in  the  deployment  of  a  system.  The  model  comparisons  between 
baseline  and  ISHM  processes  can  be  used  to  develop  a  trade-space  analysis  to  determine 
a  cost  difference  between  the  two  to  be  targeted  as  an  ISHM  development  and 
deployment  budget. 

Research  question  #2  asked:  what  is  a  reasonable  and  appropriate  scope  for  model 
development  to  establish  performance  requirements  for  ISHM  sensors  and  prognostics, 
maintenance,  and  logistic  processes?  Inclusion  of  inspection,  maintenance,  supply,  and 
health  monitoring  tasks  along  with  flight  operations  provided  reasonable  depth  in  the 
model.  Resources  and  costs  associated  with  the  aforementioned  tasks  were  easily  handled 
by  the  model  as  well.  Scope  for  this  research  was  limited  mainly  in  the  number  of 
components  modeled.  One  hundred  simulation  runs,  to  gain  statistical  significance,  took 
approximately  1  hour  to  process  on  a  standard  dual-core  PC.  There  are  no  indications  that 
inclusion  of  additional  components  to  the  model  would  provide  any  complications  other 


150 


than  increased  processing  time,  which  could  be  mitigated  with  increased  processing 
power. 

The  third  research  question  posed:  what  are  the  operational  and  maintenance  cost 
impacts  of  an  ISHM  enabled  CBM  system?  In  the  components  utilized  the  chief  cost 
driver  is  the  purchase  of  supplies.  Total  costs  for  like  size  squadrons  were  shown  to  be 
similar  between  baseline  and  ISHM  equipped  systems.  The  ISHM  systems  however  had 
higher  operational  availability  and  daily  flying  hours  that  would  allow  a  squadron  ISHM 
equipped  aircraft  to  operate  with  2  fewer  planes  and  still  exceed  the  performance  of  the 
baseline  squadron.  Additionally,  maintenance  costs  are  reduced  through  demand  versus 
time-driven  replacements  and  inspections.  The  impact  to  O&M  is  then  the  ability  to 
remove  aircraft  from  inventory  or  to  expand  into  other  roles  as  availability  and  reliability 
increase. 

It  is  suggested  that  the  lognormal  distribution  may  not  be  optimal  for  utilization  in 
determining  the  remaining  useful  life  of  components.  While  the  lognonnal  distribution 
does  eliminate  the  potential  for  negative  values  that  may  occur  under  the  left  tail  of  the 
normal  distribution,  its  skewness  under  increasing  standard  deviation,  uncertainty  in  this 
research,  likely  drives  too  many  false  alarms  for  acceptance  as  a  fielded  system. 

However,  even  with  an  elevated  number  of  false  alarms,  the  ISHM  enabled  aircraft 
provided  better  system  perfonnance  and  lower  maintenance  costs  than  a  baseline  system. 

Research  question  #4  asked  if  mean  downtime  was  a  good  measure  of  system 
performance  for  ISHM/CBM  systems.  It  was  shown  that  MDT  can  increase  with  the 
implementation  of  health  monitoring  particularly  with  maintenance  grouping.  This  is 
contrary  to  DoD  projections  of  significant  reduction  in  MDT  with  the  implementation  of 
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condition  based  maintenance  (Under  Secretary  of  Defense  (AT&L),  May  2008). 

However  with  a  poor  performing  supply  system  MDT  can  actually  increase  due  to  an 
increase  in  NMCS  time.  Therefore,  depending  on  the  scenario  an  increase  in  MDT  can  be 
both  positive  and  negative  from  a  system  performance  perspective.  This  leads  to  the 
conclusion  that  MDT  is  not  always  a  good  predictor  of  overall  system  performance. 

The  final  research  question  posed  the  question:  does  maintenance  grouping  based 
on  prognostics  improve  operational  availability,  total  downtime,  and  cost?  The  results 
show  that  maintenance  grouping  does  increase  Ao  through  a  reduction  in  total  downtime. 
While  maintenance  costs  are  reduced  with  grouping,  supply  costs  increase  as  the 
maintenance  window  increases  since  more  components  are  required  due  to  premature 
replacement  and  loss  of  useful  life. 

Significance  of  Research 

The  significance  of  this  research  lies  in  the  inclusion  of  flying,  supply,  and 
maintenance  processes  in  a  single  model  to  study  the  effects  of  integrated  systems  health 
management.  Previous  research  focused  on  these  aspects  individually,  making  limiting 
assumptions  for  those  processes  not  of  interest  and  leaving  the  interactions  between 
systems  unexplored.  Additionally,  proprietary  systems  developed  in  this  field  are  not  well 
published  for  competitive  reasons  leaving  a  gap  between  research  and  development  and 
deployment.  The  discrete  event  simulation  presented  in  this  research  provides  a  tool  for 
new  health  monitoring  techniques  to  be  analyzed  by  inserting  them  into  a  model  to 
determine  the  effect  of  their  accuracy  and  prognostics  capabilities  on  overall  system 
performance. 
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The  tool  and  simulation  outputs  present  a  capability  for  program  management  to 
use  in  determining  if  the  long-tenn  performance  increases  of  the  ISHM  system  balance 
out  the  additional  upfront  cost  of  implementation.  The  data  are  used  to  determine 
feasibility  of  solutions  and  trade-space  for  requirements  development.  Further, 
researchers  can  explore  the  interactions  of  health  monitoring,  supply  and  maintenance 
processes  to  provide  more  than  anecdotal  projections  of  savings.  Program  management 
can  use  the  tool  to  conduct  cost-benefit  analyses  when  considering  new  systems  or 
operating  procedures. 

Recommendations  for  Future  Research 

Several  recommendations  for  future  research  efforts  were  identified  during  this 
research  effort.  Further  study  should  include  exploring  repairable  systems  and  the  impact 
on  maintenance  utilization  and  repair  versus  purchase  decisions.  This  research  would  add 
further  depth  and  utility  to  the  model  as  well  as  providing  increased  interactions  between 
supply  and  maintenance  decisions.  Additionally,  the  inclusion  of  constraints  on  repair  or 
replacement  order  of  components  would  add  additional  realism  to  the  work.  This 
restriction  would  incur  delays  prior  to  the  start  of  maintenance  on  some  parts  while  work 
is  conducted  on  others  and  would  impact  grouping  decisions  and  outcomes.  As  discussed 
earlier  in  this  chapter,  remaining  useful  life  prognostics  could  be  improved  over  those 
provided  by  the  lognonnal  distribution.  It  is  suggested  that  other  RUL  formulations  be 
studied  to  potentially  further  improve  performance  and  cost. 

A  study  on  a  fielded  aircraft  with  proposed  or  deployed  health  monitoring  systems 
would  provide  further  validation  of  the  processes  and  demonstrate  real-world  impacts  to 
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decision  makers.  This  research  could  also  be  applied  to  surface  vehicles,  ships  or  other 
equipment  with  some  changes  to  operating  procedures  to  determine  the  utility  on  other 
systems  with  different  system  interactions  and  constraints.  Further,  a  cost  benefit  analysis 
of  preventive  maintenance,  and  the  associated  inspections,  for  known  failure  modes  could 
be  conducted  against  projected  cost  savings  from  ISHM.  Finally,  while  this  research 
discussed  and  provided  tools  for  trade-space  analysis,  it  may  be  possible  to  conduct  this 
type  of  simulation  as  an  optimization  problem.  Program  objectives  would  need  to  be 
established  as  well  as  constraints  on  capabilities  and  resource  allocations. 

Summary 

The  research  presented  identifies  the  impacts  of  health  monitoring  systems  on 
supply,  maintenance  and  flight  operations.  Further,  it  creates  a  tool  for  management  to 
utilize  in  the  assessment  of  implementing  health  monitoring  systems  on  new  and  legacy 
aircraft.  As  aircraft  and  other  systems  service  lives  are  extended  beyond  intended  design 
life  and  amid  shrinking  budgets  and  manpower,  it  is  necessary  to  assess  potential 
efficiencies  afforded  by  ISHM.  The  results  of  the  research  demonstrate  the  potential  to 
improve  system  availability,  reduce  cost  and  define  a  trade-space  for  ISHM  system 
assessment. 
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